Commit Graph

64 Commits

Author SHA1 Message Date
d264cf190d Add a Prometheus metrics endpoint
Add a Prometheus `/metrics` endpoint based on the modtile_collector[0]
script.
* Minor metric naming cleanup.
* Drops duplicate TileBufferReads metrics that duplicate the per-zoom values.

Related to: https://github.com/openstreetmap/operations/issues/492

[0]: https://github.com/openstreetmap/prometheus-exporters/blob/main/collectors/modtile/modtile_collector

Signed-off-by: Ben Kochie <superq@gmail.com>
2021-07-30 14:14:53 +02:00
a14924a999 Logging Improvements
* Using [`GLib Logging Framework`](https://developer.gnome.org/programming-guidelines/stable/logging.html.en) for logging
  * Created new `g_logger` function for logging
    * Outputs to `stdout`/`stderr` only when running in `foreground`
      * `stderr` for `message`, `warning`, `critical` & `error` levels
      * `stdout` for `debug` & `info` levels
        * Use `G_MESSAGES_DEBUG=all` environment to enable `debug` to print
    * Otherwise, output will be to `syslog` or `systemd journal` (when appropriate)
* Standardized usage of `{LOG_PRIORITY}: ` prefix in log messages
  * Only when using `syslog`, otherwise `GLib Logging` will take care of it
* Changed `fprintf(stderr`, `printf` & `perror` calls to `g_logger` calls
  * You might want to check them out closely to make sure I chose the right levels
  * No changes to `logging/output` were made to "`foreground`" programs (I.E. `render_*`)
* Changed `0`,`1` to `no_argument`,`required_argument` in `getopt_long`'s `long_options`
  * Fixed `renderd`'s `foreground` opt (should be `no_argument` [0] rather than `reguired_argument` [1])
* Basic test for `mod_tile` module
* ~~Extended `renderd` log priority onto Mapnik's logger~~
2021-07-28 14:46:50 +00:00
e85c7afc9e Retry a failed command after reconnecting 2021-07-14 09:20:08 +00:00
17fdb59790 Make render_list reconnect if the renderd connection is lost 2021-07-13 16:30:15 +00:00
a9e495775a include details in error log message
Be a bit more verbose in case of errors, to make finding the root cause easier
2021-07-13 16:26:36 +00:00
891ed80a83 Replace select() by poll().
select() has a well-known limit and the maximum file descriptor
that can be used (FD_SETSIZE). On Linux it is in most cases 1024.

Our renderd uses a lot of planet tiff files, so it had already
more than 1600 FDs for tiff files open. That means any new FD gets a
number bigger than that and using such FDs in select() leads to crashes
(because select() uses a fixed size bitmap).

The same can happen in mod_tile if the surounding web server is very
busy, altough it is less likely to happen there.

Changes in src/mod_tile.c:
- include poll.h
- poll timeout is an int containing miliseconds
- use "s > 0" instead of "s == 1" as success
  (although it should never be >1)
- different log message for timeout and error case

Changes in src/daemon.c:
- remove include for sys/select.h
- replace connections array by new array pfd used in poll()
- include special FDs (exit and listen) at index 0 and 1 in this array
- no longer reorganize array while iteratig through it.
  That would have led to items being skipped.
  Instead mark array slots with fixed connections with
  a negative FD. These are automatically skipped by poll.
  Reuse these slots later when new connections come in.
- rename num_connections to num_cslots (initial segment
  of the array that poll() should check) and
  num_conns (number of slots in this segment, that are
  actually used by connections; this number is only
  used in logging).
- slightly enhance debug log lines
2021-07-13 16:22:40 +00:00
48000f2679 Fixed issue were renderd does not send a response for low priority queue 2021-02-15 20:49:12 +00:00
648913dec9 Formatted code as per Linux style 2021-02-15 19:20:30 +01:00
1982516d49 Fixed problems in threading tests 2021-02-15 16:39:06 +00:00
8833bba9a2 Drop internal iniparser 2021-02-15 17:10:43 +01:00
d706d96020 Clarify copyright holders 2020-10-09 15:20:37 +00:00
8d711039c8 Merge pull request #213 from xamanu/run-for-deamon
Adhere to FHS 3.0 and use daemon under /run.
2020-09-24 11:24:05 +01:00
a2696f3901 Merge branch 'spelling-errors' of git://github.com/xamanu/mod_tile into xamanu-spelling-errors 2020-09-24 11:19:49 +01:00
de41cb1392 Adhere to FHS 3.0 and use daemon under /run. 2020-09-03 19:32:31 +00:00
3a78a312ed Fix spelling errors.
* recieve   -> receive
* accross   -> across
* determins -> determines
* reqyest   -> request

Gbp-Pq: Name spelling-errors.patch
2020-09-02 15:07:46 +00:00
a22065b8ae Move variables to prevent duplicates 2020-05-23 12:42:07 +01:00
e25bfdba1c Merge remote-tracking branch 'upstream/pull/144' 2017-01-08 19:37:17 +00:00
55c410c565 Issue #132 Fixed compilation for Mapnik 3.0.12 2016-10-07 16:51:59 +08:00
8560dfc103 Also load TTC font files 2016-10-06 21:55:35 +02:00
6b9611aaf7 Fix another compile warning 2015-05-25 23:45:08 +01:00
3a4b4712eb Add options support into memcached and rados stores
This matches the support in other backing stores but is untested.
2015-05-25 23:02:48 +01:00
8307d19ab7 Fix lots of compilation warnings 2015-05-25 21:58:27 +01:00
9e12eeaa5a Merge pull request #103 from jburgess777/mapnik3
Add support for mapnik-3.0.0rc2
2015-05-25 21:03:11 +01:00
bd463dc85d Fix image_view usage in metatiling
The previous code ended up with the complete metatile being
included in every tile.
2015-05-25 18:15:01 +01:00
13a7c55688 fix compile with older mapnik 2015-04-16 20:18:14 +02:00
7816ea45d1 fix missing symbol 2015-04-16 18:54:42 +02:00
46af34d562 fix compile with latest mapnik 3.x 2015-04-16 12:04:03 +02:00
42228dc076 scan & register opentype fonts 2015-03-13 00:39:38 +08:00
6aa08808a2 fix compile with latest mapnik - refs mapnik/mapnik#2712 2015-02-20 10:31:08 -08:00
c3a6339b5f fix compile against mapnik 3.x at 884de5a0440d5521e4ca50177e10a6584610ffd6 2015-02-12 15:01:25 -08:00
b8bf98107f include needed headers for mapnik 2014-08-13 08:37:09 -07:00
ccbc9e7a2b fix usage of mapnik::parameters 2014-08-13 08:36:36 -07:00
1d43b86762 Merge branch 'master' of github.com:openstreetmap/mod_tile 2014-08-13 08:11:35 -07:00
b76d7a744b resize existing map rather than creating a new one 2014-08-13 08:11:31 -07:00
34dbe66acb Merge pull request #82 from Zverik/retina
SCALE option to produce retina tiles
2014-07-09 12:56:12 -06:00
7e9abfbb29 SCALE option to produce retina tiles 2014-07-09 18:23:41 +04:00
6600d38a54 Extend autoconf and automake to use an installed libiniparser
This helps systems with newer (and therefore relying on libiniparser) samba versions installed.
2014-05-29 08:56:52 +02:00
122adae7fb osx compile fixes 2014-05-23 14:09:13 -07:00
dcc919c5e4 support Mapnik 3.x 2014-05-20 17:10:00 -07:00
ebfd8aafbb support Mapnik 3.x 2014-05-20 17:08:12 -07:00
9d4e10e7c0 Parameterization options and mime type of renderd protocol 3 were not passed to slave threads
Expand some socket debuging information
2014-05-18 11:32:22 -06:00
13da3afe70 render_list was not blocking on requests and would swamp renderd with requests 2013-12-24 01:03:20 +01:00
4f531b682a Replace malloc+memset by calloc in render_expired.c
This improves startup time of render_expired, when using --max-zoom=20, as a
2GB array is initialized to 0.
2013-12-08 00:05:07 +01:00
66e5538dc4 fix gcc errors 2013-12-05 17:47:04 +01:00
5dffe058e6 fix gcc errors 2013-12-05 16:35:49 +01:00
e5d618459e fix gcc errors 2013-12-05 16:19:05 +01:00
681fbd4eb2 Limit sscan in URL parsing to not overflow the allocated bufferes 2013-10-23 21:03:55 -06:00
c0bc71162d Fix issues found via coverity scan 2013-10-23 20:09:56 -06:00
e2c4b62e67 Fix issues found via coverity scan 2013-10-23 20:09:27 -06:00
db982683ad Fix issues found via coverity scan 2013-10-23 20:08:33 -06:00