The sscanf expression to parse the URL was too greedy, lumping the /dirty and /status into the file extension,
causing the tile layers being rejected due to a wrong file extension.
In the commit to add support for other file extensions other than png,
AddTileConfig configuration was broken due to a typo.
Bug reported by Sven Geggus and Frederik Ramm
Make it possible to serve tiles other than raster png files,
e.g. jpeg or javascript vector tiles.
This adds a configuration parameter to tile layers specifying the
file extension and mimetype. The default remain image/png.
If you have several styles served from the same mod_tile, it might be desirable to have
different planet-import-complete dates. This can for example be used if the style sheet changes
and a complete re-render is needed.
It first checks the planet-import-complete file in the tile directory of the style and if the file doesn't
exist checks in the root tile directory.
If mod_tile serves many different tile styles, it might be interesting how havily various
layers are used and which layers particularly have many 404 errors due to not rendering fast enough.
Tile scrapers can often overburden the resources available to a tileserver by requesting large areas at high zooms.
Rather than banning these users from the tileserver, this patch adds the ability to mod_tile to automatically throttle
these clients to acceptable limits.
It uses a token bucket implementation with configurable bucket size and top up rate. Accounting is done in a hashtable
with a small possibility of ip hashtable clashes
The code also supports whitelisting a few ip addresses, such as proxy servers or CDN nodes, although at the moment
there is no way to configure these ip's other than through adding them to the code and recompiling.
This allows to have hostname aliases (e.g. cache.tile.openstreetmap.org) to become more (or less)
cache friendly, to tune the trade off between up-to-dateness and minimizing network resources
In addition to the request queue and the dirty queue, there are now also a requestPrio and requestBulk queue.
The rendering order now is first render requests from requestPrio, then from request followed by the
dirty queue and finally if no other requests are queued, render from the requestBulk queue.
RequestPrio, Request and RequestBulk all block, whereas Diry immediately returns with NotDoneYet.
This also changes mod_tile to submit requests that if not rendered in time would result in 404 errors as high priority.
prioBulk should be useful for things like rerendering all the outdated tiles in the background, but this patch
does not include those changes.
This patch reports two properties of the served tiles. First it reports how many tiles are returned with which response code, i.e.
how many tiles get return with http code 200, 304, 404 or 500. Secondly, it reports if tiles are returned fresh from disk, fresh
after being rendered on the spot, old without attempting to render, old after timing out waiting for the tile to be rendered.
The stats get reported through a special page at "http://servername/mod_tile".
Caveat: In order to report the stats of a global counter, a mutex is needed that serialises access to that counter. This can have
some performance effects, although it is attempted to hold this lock as short as possible. Measuring the effect with apache bench,
running for 1 million tile requests at a concurrency level of 10, this gave about a slowdown of 1%. This should be pretty much
worst case, as apache bench always requests the same tile, so that is all in cache. With this, it still achieves more than 5800
requests per second at 50Mb/s on a dual core laptop. On different hardware, this can obviously differ.
Stats reporting can be disabled with a config setting in mod_tile.conf
The locking code is based on mod_example_ipc.c
The initialisation of the per zoom level cache expire table was done in the wrong
section of the configuration parsing. This meant the results were order dependent in statements in mod_tile.conf,
as some of the values might not have been correctly set at that time of calculating the table.
Achieving a high cache hit ratio is both beneficial to clients and the server. For the clients it is
reduced loading time and for the server reduced resources used. To achieve this, cache expiries should be set
as long as possible. However, this conflicts with wanting up to date tiles, especially with minutely renderings.
As you can't predict the future, one can only use a bunch of heuristics.
With this patch, the tile expiry heuristic is based on three factors. The frequency of planet updates (the heuristic used so far),
the zoom level and the last modified time. The idea behind the zoom level heuristic is that low zoom tiles don't change noticably
very often, and so can use longer cache expiries than high zoom tiles. The last modified heuristic is based on the assumption
that a tile that hasn't changed for a while is more likely to stay unchanged than one that has recently been modified.
The heuristics have a bunch of tunables that can be set in the apache config of mod_tile
* -t|--tile-dir option for render_list and convert_meta
* renderd reads it from renderd.conf section "[renderd]" setting "tile_dir"
* mod_tile reads from Apache conf ModTileTileDir directive
There are still some places in store.c left, where HASH_DIR is used directly. More fixes needed.
Made renderd more configurable at runtime: socketname, num_threads and the mapnik settings plugins_dir, font_dir, and font_dir_recurse are now available through renderd.conf
There should be only one incompatible change: the [Default] section in the config is now in all lowercase "[default]" because the parser lib gives section and key names back in lowercase
