Commit Graph

520 Commits

Author SHA1 Message Date
fce4cb8582 Added experimental python implementation of renderd. 2021-08-04 11:14:09 +00:00
fdf94e683f Added flawfinder code check 0.6 2021-08-04 11:03:22 +00:00
c683c5569b Added compilation instructions for several distros 2021-08-03 19:16:48 +02:00
4ea8729e14 Added build & test on CentOS 7
* Requires building `Mapnik`
  * Which requires these `EPEL` repository packages
    * `boost169-devel` + dependencies
    * `gdal-devel` + dependencies
    * `proj-devel` + dependencies
* Caches `Mapnik` build directory
  * Takes about 20-30 minutes without cache
  * Takes about 5-10 minutes with cache
* Runs on `push` to `centos7` branch & `pull_request`
* Includes very basic build documentation
2021-08-03 07:43:47 +02:00
7c4081dc81 Added build & test on Fedora 34
* Takes about 3-5 minutes

**Also:**
* Split up `Lint` and `Build & Test`
  * Into independent workflows
* Changed `.tar` compression to `GZip`
  * `--zstd` not available in `CentOS 7`
  * Also excluded `.git` & `.gitignore`
* Created `Provisioning` local action
  * Combining update & install steps, etc.
* Added libraries to `Ubuntu` build dependencies
  * `libmemcached-dev` & `librados-dev`
  * In order to test for successful compilation
* ~~Reduced `build-archive` retention to 1 day~~
* Updated `ax_pthread` macro
  * While trying to resolve `apxs` failure
  * http://www.gnu.org/software/autoconf-archive/ax_pthread.html
2021-08-02 17:03:19 +02:00
ffda917722 Convert -pthread to -Wc,-pthread when invoking apxs 2021-08-02 14:44:28 +02:00
a642a232f0 Fix .gitignore ignoring too much
* `Makefile.am` ignored by `Makefile*`
* `etc/renderd/*` ignored by `renderd`
2021-08-02 10:21:13 +02:00
cab62c1612 Reduce redundant CI configuration by using local actions 2021-07-30 16:45:05 +02:00
0f017f1ae8 Completed list of authors with recent contributors 2021-07-30 12:16:15 +00:00
d264cf190d Add a Prometheus metrics endpoint
Add a Prometheus `/metrics` endpoint based on the modtile_collector[0]
script.
* Minor metric naming cleanup.
* Drops duplicate TileBufferReads metrics that duplicate the per-zoom values.

Related to: https://github.com/openstreetmap/operations/issues/492

[0]: https://github.com/openstreetmap/prometheus-exporters/blob/main/collectors/modtile/modtile_collector

Signed-off-by: Ben Kochie <superq@gmail.com>
2021-07-30 14:14:53 +02:00
a7baee9033 Relied on leaflet package for CI 2021-07-29 23:48:54 +00:00
27c470c3eb Improved configuration files 2021-07-29 22:57:26 +00:00
6f93441f60 Adjusted installation instructions 2021-07-29 22:57:21 +00:00
0169d767a1 Install, configure & start apache & renderd, test for mod_tile.so 2021-07-29 10:01:20 +02:00
f28cda9920 Easier installation instructions 2021-07-28 14:46:55 +00:00
a14924a999 Logging Improvements
* Using [`GLib Logging Framework`](https://developer.gnome.org/programming-guidelines/stable/logging.html.en) for logging
  * Created new `g_logger` function for logging
    * Outputs to `stdout`/`stderr` only when running in `foreground`
      * `stderr` for `message`, `warning`, `critical` & `error` levels
      * `stdout` for `debug` & `info` levels
        * Use `G_MESSAGES_DEBUG=all` environment to enable `debug` to print
    * Otherwise, output will be to `syslog` or `systemd journal` (when appropriate)
* Standardized usage of `{LOG_PRIORITY}: ` prefix in log messages
  * Only when using `syslog`, otherwise `GLib Logging` will take care of it
* Changed `fprintf(stderr`, `printf` & `perror` calls to `g_logger` calls
  * You might want to check them out closely to make sure I chose the right levels
  * No changes to `logging/output` were made to "`foreground`" programs (I.E. `render_*`)
* Changed `0`,`1` to `no_argument`,`required_argument` in `getopt_long`'s `long_options`
  * Fixed `renderd`'s `foreground` opt (should be `no_argument` [0] rather than `reguired_argument` [1])
* Basic test for `mod_tile` module
* ~~Extended `renderd` log priority onto Mapnik's logger~~
2021-07-28 14:46:50 +00:00
1ed85f3f8d Documented installation of Debian and Ubuntu packages from official sources 2021-07-27 12:22:25 +00:00
b1091e201b Switched CI from Travis to Github Actions 2021-07-27 14:06:24 +02:00
865a44e63f Explicitly mention Debian packages for config
Explicitly state that Debian helper scripts require packages and configuration section does not apply when building from source.
Fixes #241 misunderstanding.
2021-07-20 15:28:35 +02:00
e85c7afc9e Retry a failed command after reconnecting 2021-07-14 09:20:08 +00:00
17fdb59790 Make render_list reconnect if the renderd connection is lost 2021-07-13 16:30:15 +00:00
a9e495775a include details in error log message
Be a bit more verbose in case of errors, to make finding the root cause easier
2021-07-13 16:26:36 +00:00
891ed80a83 Replace select() by poll().
select() has a well-known limit and the maximum file descriptor
that can be used (FD_SETSIZE). On Linux it is in most cases 1024.

Our renderd uses a lot of planet tiff files, so it had already
more than 1600 FDs for tiff files open. That means any new FD gets a
number bigger than that and using such FDs in select() leads to crashes
(because select() uses a fixed size bitmap).

The same can happen in mod_tile if the surounding web server is very
busy, altough it is less likely to happen there.

Changes in src/mod_tile.c:
- include poll.h
- poll timeout is an int containing miliseconds
- use "s > 0" instead of "s == 1" as success
  (although it should never be >1)
- different log message for timeout and error case

Changes in src/daemon.c:
- remove include for sys/select.h
- replace connections array by new array pfd used in poll()
- include special FDs (exit and listen) at index 0 and 1 in this array
- no longer reorganize array while iteratig through it.
  That would have led to items being skipped.
  Instead mark array slots with fixed connections with
  a negative FD. These are automatically skipped by poll.
  Reuse these slots later when new connections come in.
- rename num_connections to num_cslots (initial segment
  of the array that poll() should check) and
  num_conns (number of slots in this segment, that are
  actually used by connections; this number is only
  used in logging).
- slightly enhance debug log lines
2021-07-13 16:22:40 +00:00
acb11808d6 Update README.rst
one letter is missing
2021-03-21 08:26:14 +00:00
48000f2679 Fixed issue were renderd does not send a response for low priority queue 2021-02-15 20:49:12 +00:00
8f1f3ad5ff Applied shell code formatting 2021-02-15 18:57:51 +00:00
22d4cc1335 Moved munin plugin to utils directory 2021-02-15 18:39:59 +00:00
9b6124790a Added code format check to CI pipeline 2021-02-15 19:20:30 +01:00
648913dec9 Formatted code as per Linux style 2021-02-15 19:20:30 +01:00
a125f9c7f3 Formatted Makefile 2021-02-15 19:20:30 +01:00
1982516d49 Fixed problems in threading tests 2021-02-15 16:39:06 +00:00
8833bba9a2 Drop internal iniparser 2021-02-15 17:10:43 +01:00
f7f42b8d9a Fixed CI pipeline 2021-02-14 19:39:59 +00:00
6ca983408a Added Linux as dependency 2021-02-14 10:31:41 +00:00
e1b3df907c Fix copyright statement in README 2021-02-11 21:17:40 +00:00
a3f4230df6 Update README instructions 2020-10-27 10:26:35 +00:00
d706d96020 Clarify copyright holders 2020-10-09 15:20:37 +00:00
889a45b5e0 Use /var/cache instead of /var/lib for tiles 2020-10-09 07:52:15 +00:00
557e419d07 Moved scripts to utils directory 2020-10-09 07:49:06 +00:00
41083a23a4 Moved dist config files 2020-10-09 07:48:58 +00:00
676d011f2d Replace example map 2020-10-08 09:26:12 +00:00
ba0c7f66a8 remove meta2tile 2020-10-08 09:26:04 +00:00
a49a33e401 Remove debian packaging - it's being handled salsa.debian.org/debian-gis-team/libapache2-mod-tile 2020-10-08 09:24:02 +00:00
bb1782300d Drop experimental Python implementation 2020-10-08 09:23:55 +00:00
1d8138a16b Merge pull request #174 from MaZderMind/patch-2
replicate-seqences-tool has long been moved
0.5
2020-09-28 07:43:13 +00:00
bdbd80e415 Merge pull request #168 from Ircama/render_list.1-options
Added .SH OPTIONS to docs/render_list.1
2020-09-28 07:42:30 +00:00
0f44afa34e Merge pull request #187 from realgo/config-examples
Conf typos fixed, merging confs.
2020-09-28 07:40:39 +00:00
8d711039c8 Merge pull request #213 from xamanu/run-for-deamon
Adhere to FHS 3.0 and use daemon under /run.
2020-09-24 11:24:05 +01:00
a939922bb5 Merge branch 'xamanu-spelling-errors' 2020-09-24 11:20:06 +01:00
a2696f3901 Merge branch 'spelling-errors' of git://github.com/xamanu/mod_tile into xamanu-spelling-errors 2020-09-24 11:19:49 +01:00