In addition to the request queue and the dirty queue, there are now also a requestPrio and requestBulk queue.
The rendering order now is first render requests from requestPrio, then from request followed by the
dirty queue and finally if no other requests are queued, render from the requestBulk queue.
RequestPrio, Request and RequestBulk all block, whereas Diry immediately returns with NotDoneYet.
This also changes mod_tile to submit requests that if not rendered in time would result in 404 errors as high priority.
prioBulk should be useful for things like rerendering all the outdated tiles in the background, but this patch
does not include those changes.
This patch reports two properties of the served tiles. First it reports how many tiles are returned with which response code, i.e.
how many tiles get return with http code 200, 304, 404 or 500. Secondly, it reports if tiles are returned fresh from disk, fresh
after being rendered on the spot, old without attempting to render, old after timing out waiting for the tile to be rendered.
The stats get reported through a special page at "http://servername/mod_tile".
Caveat: In order to report the stats of a global counter, a mutex is needed that serialises access to that counter. This can have
some performance effects, although it is attempted to hold this lock as short as possible. Measuring the effect with apache bench,
running for 1 million tile requests at a concurrency level of 10, this gave about a slowdown of 1%. This should be pretty much
worst case, as apache bench always requests the same tile, so that is all in cache. With this, it still achieves more than 5800
requests per second at 50Mb/s on a dual core laptop. On different hardware, this can obviously differ.
Stats reporting can be disabled with a config setting in mod_tile.conf
The locking code is based on mod_example_ipc.c
The initialisation of the per zoom level cache expire table was done in the wrong
section of the configuration parsing. This meant the results were order dependent in statements in mod_tile.conf,
as some of the values might not have been correctly set at that time of calculating the table.
Achieving a high cache hit ratio is both beneficial to clients and the server. For the clients it is
reduced loading time and for the server reduced resources used. To achieve this, cache expiries should be set
as long as possible. However, this conflicts with wanting up to date tiles, especially with minutely renderings.
As you can't predict the future, one can only use a bunch of heuristics.
With this patch, the tile expiry heuristic is based on three factors. The frequency of planet updates (the heuristic used so far),
the zoom level and the last modified time. The idea behind the zoom level heuristic is that low zoom tiles don't change noticably
very often, and so can use longer cache expiries than high zoom tiles. The last modified heuristic is based on the assumption
that a tile that hasn't changed for a while is more likely to stay unchanged than one that has recently been modified.
The heuristics have a bunch of tunables that can be set in the apache config of mod_tile
* -t|--tile-dir option for render_list and convert_meta
* renderd reads it from renderd.conf section "[renderd]" setting "tile_dir"
* mod_tile reads from Apache conf ModTileTileDir directive
There are still some places in store.c left, where HASH_DIR is used directly. More fixes needed.
Made renderd more configurable at runtime: socketname, num_threads and the mapnik settings plugins_dir, font_dir, and font_dir_recurse are now available through renderd.conf
There should be only one incompatible change: the [Default] section in the config is now in all lowercase "[default]" because the parser lib gives section and key names back in lowercase
