This is the draft of a topic to be included in the documentation set for Foswiki 1.1, as described in Tasks.Item8706. It is being drafted here to involve as many people as possible in writing the page, so we document this complex feature clearly.
See
FoswikiCache for background
TODO:
-
Explain 'dirty areas'
- Explain where and when namespaces matter
Page Caching
By default, Foswiki pages are delivered to the browser using HTML, which is regenerated from the data stored in Foswiki each time the page is requested. Page caching is the process of storing (caching) this HTML on the server, as and when it is generated, so that the next time someone asks for the same page it can be delivered from the cache instead of having to be re-rendered from scratch. Page caching can significantly improve the performance of a Foswiki site, especially where most requests are views (this is normal on most sites). Since version 1.1, Foswiki has built-in page caching.
For most Foswiki installations you can enable page caching by simply selecting the
{Cache}{Enabled}
option in the Tuning section of
configure
. However to get the most out of the cache, you need to understand what it is doing, and may want to tune the cache options for your site.
When Foswiki generates a page, it may refer to a lot of content stored in the Foswiki database. For example, a topic 'MyTopic' may be rendered differently depending on a setting in WebPreferences, which is itself a topic. If WebPreferences were to be edited, then the cache of MyTopic would have to be regenerated, because the setting it depends on might have been changed during the edit. For an average page there are a large number of these
dependencies that have to be tracked. This tracking has to be fast, so the cache uses a database engine to record the dependencies and save the rendered pages.
Using the Cache
By its very nature, a cache should be invisible to the end user. However there are some things users may have to know to get the best out of the Foswiki cache.
Topics that always change
Some other topics change automatically whenever
any topic in the same web is saved. These are usually topics that are generated by searches over the content. You can list the names of these topics a
WEBDEPENDENCIES
web preference setting. This is a comma-separated list of topics names.
Dirty Areas
Sometimes caching complete pages is too coarse-grained. There may be parts of a page that change frequently, while the rest of the same page never change. In this case the author of the topic can tell the cache not to save certain parts of it, called
dirty areas. Dirty areas are marked in the topic using the
<dirtyarea>...</dirtyarea>
tags. Foswiki markup within a dirty area is stored unrendered in the cache, and only rendered when the topic is served from the cache. For example,
<dirtyarea> %GMTIME{"$iso"}% </dirtyarea>
ensures that the cache will never store the time, forcing it to be re-computed every time the page is served.
Use of dirty areas will defeat compression of the pages that use them, so should be used sparingly.
Configuring the Cache
Choosing a database engine
The database engine used by the cache is selected using the
{Cache}{CacheManager}
EXPERT setting in
configure
.
- Use
Foswiki::Cache::FileCache
for long term caching (where topics change very infrequently). Cached pages will be stored on disk. This is the default cache type. The required CPAN libraries are included with Foswiki, so it should work "out-of-the-box", even on hosted sites.
-
Foswiki::Cache::BDB
uses the fast and freely available Berkeley DB, and is the recommended cache type for most sites. However it requires the Berkeley DB to be installed on the server, and requires the additional Perl module BerkeleyDB
(available from CPAN).
- Use
Foswiki::Cache::Memcached
for distributed caching on high end sites. Look up memcached
in your favourite search engine for more information on how this powerful tool works. This option requires the Cache::Memcached
Perl module, available from CPAN.
- Use
Foswiki::Cache::MemoryLRU
for an in-memory LRU (least recently used) cache. This is only useful when used in conjunction with a persistent perl back end, such as mod_perl
or fastcgi
. Pages will be cached for the lifetime of each persistent backend process (but are not shared between different backend instances).
The
{Cache}{MetaCacheManager}
EXPERT setting in
configure
is used to select the database implementation used to store cache meta-data (data about the cache). This data needs to be accessed as fast as possible.
-
Foswiki::Cache::DB_File
uses plain files to store the cache. This is the default and works "out-of-the-box".
-
Foswiki::Cache::BDB
uses the Berkeley DB, which is recommended, but requires additional libraries to be installed (see above).
While the cached pages might be safely discarded in a page cache, meta data about page dependencies must be stored reliably. That's why only DB_File and Berkeley DB are selectable for the meta data cache.
Namespaces
Sometime you may want to use a database that is shared with other Foswiki installs. In this case you need to be able to distinguish data that is stored for this install versus other users of the database. To do this you can set the
{Cache}{Namespace}
EXPERT setting in
configure
to a name that is unique for the configured system.
- Is the above true?
- Which engines are these required for....
- Why isn't the namespace used in the variation key?
Tuning
A number of options exist to help you tune the cache so that it works better for your specific content.
Selecting topics to ignore
Some topics change so infrequently that it's worth telling the cache about them so they can be ignored when computing dependencies. The
{Cache}{DependencyFilter}
EXPERT setting in
configure
defines a regular expression that, when matched by the name of a topic, exclude the topic from dependencies in the cache. This helps to reduce the number of topics a page depends on, but at a higher risk that you may experience unwanted caching effects. These effects may be seen where a cached page is delivered that does not reflect changes made in a topic
that matched the filter.
Topics that always change
Some other topics change automatically whenever
any topic in the same web is saved. These are usually topics that are generated by searches over the content. You can list the names of these topics in the
{Cache}{WebDependencies}
EXPERT setting in
configure
. Whenever any topic in the same web is saved, the cached versions of the listed topics are removed from the cache as well.
Web dependencies can also be specified using the
WEBDEPENDENCIES
web preference setting. If
WEBDEPENDENCIES
is set, it overrides the setting of
{Cache}{WebDependencies}
for that web.
Compression
Modern browsers can decompress content encoded using gzip compression. You can save a lot of network bandwidth by compressing pages, at the expense of some server performance spent on compressing the pages. Foswiki supports content compression when the
{HttpCompress}
option is enabled in
configure
, even when the cache is not enabled.
Content compression makes even more sense when used with the cache, as the cache by default stores pages on disk already compressed, so serving a compressed page is very fast. However if you don't have
Compress::Zlib
, or you're not using
{HttpCompress}
and do not care about disk space, you can disable this compression by disabling the
{Cache}{Compress}
EXPERT option in
configure
.
Note that only pages without any 'dirty areas' will be compressed. Any other page will be transmitted uncompressed.
Technical notes
Working with browsers
The Foswiki cache works closely with the browser to optimise the use of network bandwidth. Whenever a page is cached, Foswiki also stores its
etag, computed based on the time it was added to the server cache. The
Etag
and
Last-Modified
headers will be added to the response. So whenever a page is requested again, using a request that includes the
If-None-Match
and/or
If-Modified-Since
request headers, then Foswiki will answer with a
304 - Not modified message
and an empty body. This tells the browser to reuse the page stored in its own client-side cache.
Identifying what is in the cache
Depending on the values of a number of different parameters, a generated page may have very different output.
For example, depending on the user who is logged in, a page might be displayed very differently.
The cache has to record a this sort of environment information in order to correctly identify pages in the cache.
- The server serving the request (HTTP_HOST)
- The port number of the server serving the request (HTTP_PORT)
- The language of the current session, if any
- All session parameters EXCEPT:
- All those starting with an underscore
- VALIDATION
- REMEMBER
- FOSWIKISTRIKEONE.*
- VALID_ACTIONS.*
- BREADCRUMB_TRAIL
- All the request parameters EXCEPT:
- All those starting with an underscore
- refresh
- foswiki_redirect_cache
- logout
- style.*
- switch.*
- topic
Dependencies
- {HttpCompress} and {Cache}{Compress} depend on
Compress::Zlib
- Cache::FileCache,>=0,cpan,Required
- Cache::Memcached,>=0,cpan,Optional
- Cache::MemoryCache,>=0,Optional
- DB_File,>=0,cpan,Optional
- BerkeleyDB,>=0,perl,Optional, used by the BDB CacheManager and MetaCacheManager