Commit Graph

18 Commits

Author SHA1 Message Date
d264cf190d Add a Prometheus metrics endpoint
Add a Prometheus `/metrics` endpoint based on the modtile_collector[0]
script.
* Minor metric naming cleanup.
* Drops duplicate TileBufferReads metrics that duplicate the per-zoom values.

Related to: https://github.com/openstreetmap/operations/issues/492

[0]: https://github.com/openstreetmap/prometheus-exporters/blob/main/collectors/modtile/modtile_collector

Signed-off-by: Ben Kochie <superq@gmail.com>
2021-07-30 14:14:53 +02:00
a14924a999 Logging Improvements
* Using [`GLib Logging Framework`](https://developer.gnome.org/programming-guidelines/stable/logging.html.en) for logging
  * Created new `g_logger` function for logging
    * Outputs to `stdout`/`stderr` only when running in `foreground`
      * `stderr` for `message`, `warning`, `critical` & `error` levels
      * `stdout` for `debug` & `info` levels
        * Use `G_MESSAGES_DEBUG=all` environment to enable `debug` to print
    * Otherwise, output will be to `syslog` or `systemd journal` (when appropriate)
* Standardized usage of `{LOG_PRIORITY}: ` prefix in log messages
  * Only when using `syslog`, otherwise `GLib Logging` will take care of it
* Changed `fprintf(stderr`, `printf` & `perror` calls to `g_logger` calls
  * You might want to check them out closely to make sure I chose the right levels
  * No changes to `logging/output` were made to "`foreground`" programs (I.E. `render_*`)
* Changed `0`,`1` to `no_argument`,`required_argument` in `getopt_long`'s `long_options`
  * Fixed `renderd`'s `foreground` opt (should be `no_argument` [0] rather than `reguired_argument` [1])
* Basic test for `mod_tile` module
* ~~Extended `renderd` log priority onto Mapnik's logger~~
2021-07-28 14:46:50 +00:00
a9e495775a include details in error log message
Be a bit more verbose in case of errors, to make finding the root cause easier
2021-07-13 16:26:36 +00:00
891ed80a83 Replace select() by poll().
select() has a well-known limit and the maximum file descriptor
that can be used (FD_SETSIZE). On Linux it is in most cases 1024.

Our renderd uses a lot of planet tiff files, so it had already
more than 1600 FDs for tiff files open. That means any new FD gets a
number bigger than that and using such FDs in select() leads to crashes
(because select() uses a fixed size bitmap).

The same can happen in mod_tile if the surounding web server is very
busy, altough it is less likely to happen there.

Changes in src/mod_tile.c:
- include poll.h
- poll timeout is an int containing miliseconds
- use "s > 0" instead of "s == 1" as success
  (although it should never be >1)
- different log message for timeout and error case

Changes in src/daemon.c:
- remove include for sys/select.h
- replace connections array by new array pfd used in poll()
- include special FDs (exit and listen) at index 0 and 1 in this array
- no longer reorganize array while iteratig through it.
  That would have led to items being skipped.
  Instead mark array slots with fixed connections with
  a negative FD. These are automatically skipped by poll.
  Reuse these slots later when new connections come in.
- rename num_connections to num_cslots (initial segment
  of the array that poll() should check) and
  num_conns (number of slots in this segment, that are
  actually used by connections; this number is only
  used in logging).
- slightly enhance debug log lines
2021-07-13 16:22:40 +00:00
648913dec9 Formatted code as per Linux style 2021-02-15 19:20:30 +01:00
d706d96020 Clarify copyright holders 2020-10-09 15:20:37 +00:00
3a78a312ed Fix spelling errors.
* recieve   -> receive
* accross   -> across
* determins -> determines
* reqyest   -> request

Gbp-Pq: Name spelling-errors.patch
2020-09-02 15:07:46 +00:00
8307d19ab7 Fix lots of compilation warnings 2015-05-25 21:58:27 +01:00
5dffe058e6 fix gcc errors 2013-12-05 16:35:49 +01:00
681fbd4eb2 Limit sscan in URL parsing to not overflow the allocated bufferes 2013-10-23 21:03:55 -06:00
99357a53ab Improve backwards and forward compatibility for renderd protocol v3/v2 2013-10-20 16:03:32 -06:00
e70bfafa20 Extend the storage backends to support options 2013-10-13 23:25:14 -06:00
5941e23ced Support options for tile layers in mod_tile.
If a tile layer has options enabled, the URL changes to

http://localhost/style/options/z/x/y.ext
2013-10-13 23:23:10 -06:00
fdc317a8f3 Update mod_tile to renderd protocol to version 3.
Include an "options" string and a "mimetype" string.
2013-10-13 23:22:08 -06:00
0ce9f43437 Allow to configure different aspect ratios of tiles.
For example for Plate Carree which has a 2x1 ratio
or OSGB which can use a 1x2 ratio
2013-08-16 02:35:31 -06:00
4c2faccb12 Improve CORS support
* Only treat OPTIONS as preflight if Access-Control-Request-Method  is set
* Only send Access-Control-Allow-Headers for preflight requests
* Don't send a bogus Access-Control-Allow-Headers value
2013-08-13 09:33:04 +01:00
1b4c56d8ed Add another priority level (RenderLow) to accommodate rerenders after style changes
In addition to Render and RenderPrio, add another priority level of RenderLow.

The idea is that if you rerender all tiles by touching planet-import-complete and
thereby likely overloading the server's rendering capacity you want to prioritize rerendering
of tiles that were explicitly marked dirty due to data changes over rerendering
due to planet-import-complete updates.

This is achieved by splitting rendering requests by if they are "old", or "very old".
As the tile expiry sets back the modification date many years, mod_tile splits
rendering requests based on how old the tiles are. If they are no more than a
certain threshold (currently set to one year) old, they are rendered with lower priority.
2013-08-10 23:05:28 -06:00
273973ed03 Clean up mod_tile directory by moving source files into a common src/ sub directory
also move includes into includes/
2013-08-09 18:46:10 -06:00