Commit Graph

645 Commits

Author SHA1 Message Date
0cc76cb72f Cleaning up tests (#424) 2024-03-27 17:58:11 -07:00
7f91d53ec3 Prevent duplicate renderd sections (#421)
In order to remove any confusion if the user has duplicate `renderd` sections in their `renderd.conf` file.

Currently with a configuration like this one:
```ini
[renderd]
pid_file=/run/renderd/renderd.pid
socketname=/run/renderd/renderd.sock
stats_file=/run/renderd/renderd.stats
tile_dir=/var/cache/renderd/tiles

[renderd0]
pid_file=/run/renderd/renderd0.pid
socketname=/run/renderd/renderd0.sock
stats_file=/run/renderd/renderd0.stats
tile_dir=/var/cache/renderd/tiles
```
only the second `renderd` section (`renderd0`) will actually be used.

After this patch is merged, an error message will be shown and the application will exit.
2024-03-24 21:25:57 -07:00
aebb506485 Prevent socketnames in renderd.conf exceeding maximum allowed length (#422)
The maximum allowed length of a socket file name is 108:
https://www4.cs.fau.de/Services/Doc/C/libc.html#SEC189

Exceeding this length will likely lead to difficult to troubleshoot behavior.
2024-03-24 20:55:27 -07:00
2ce51df526 Test error output in catch tests (#419) 2024-03-23 21:58:30 -07:00
743cf651b1 Follow up to #190 for render_list (#417)
* Fixed `render_list` build needs `libm`
* Added tests
* Updated man page
* Created function to set double
* Cleaned up / optimized a bit
* Removed `--dont-render` option
2024-03-21 13:22:02 -07:00
3621428efd Enable rendering between geo-coords instead of tile-numbers (#190)
Instead of giving tile-numbers to render in a specific zoomlevel i wanted to give a gectangle defined by geo-coords and render all zoomlevels at once
2024-03-21 11:53:31 -07:00
2a4532f002 Improve render_expired/render_list test coverage (#415) 2024-03-19 09:15:41 -07:00
3ad89950fc OpenSUSE doesn't seem to report coverage (#414)
Disabling it for now
2024-03-16 19:52:36 -07:00
b7118780dc Added continue-on-error for experimental distros (#413)
So that workflows will still show as passing when issues arise.

* Build & Test
  * debian:unstable
  * fedora:rawhide
  * ubuntu:devel
* Docker Image Build
  * debian-unstable/debian-unstable-autotools
  * fedora-rawhide
  * opensuse-tumbleweed
  * ubuntu-devel/ubuntu-devel-autotools
2024-03-16 16:55:23 -07:00
13a198d06d Updated Docs/Dockerfiles/GitHub actions after switching CMake to relative install paths (#412) 2024-03-16 14:57:19 -07:00
4f04956289 Fix installation when using DESTDIR. (#411)
The Debian package build failed after switching to CMake because it uses `DESTDIR` to install under `debian/tmp`.
2024-03-16 10:01:44 -07:00
21713d0319 Fix/improve X-Forwarded-For/throttling testing (#410)
* Previously it was not actually sending the header
* Now runs in serial, regardless of parallel specification
* Now runs only once, rather than for each map layer

_Also_:
* Removed no-longer used `storage_mutex`
  * Was only used for Apache HTTP Server < v2.4
2024-03-15 10:11:37 -07:00
d67a672750 Add macOS 14 to build and test CI workflow (#409)
Available since January 30th, 2024:
https://github.blog/changelog/2024-01-30-github-actions-macos-14-sonoma-is-now-available/
2024-03-14 13:56:32 -07:00
c3d9bd2d98 CPack Updates (#408)
* Add project description and URL for CPack packaging
* Address cppcheck issues
2024-03-13 22:04:58 -07:00
a404c810b1 Remove support for Apache HTTPD < 2.4 (#407)
* `v2.4` was released in `2012`
* `v2.2` has not been updated since `2017`
2024-03-13 20:46:03 -07:00
d9dbf82524 Add CPack packaging to GitHub Actions (#406) 2024-03-12 22:58:15 -07:00
0aa7607d7b Updates to "full" Docker containers (#404)
* To help with testing rendering with real data and styles
2024-03-08 15:38:46 -07:00
c9e7dd8845 Update man pages for render_expired/render_list (#403)
_Also updated all render\_* apps to_:
* Use `MAX_ZOOM` rather than hard-coded values for `--max-zoom`
* Print `metatile` rather than `meta tile`
* Add missing `--help` output to `render_expired` & `render_list`
* Remove redundant `--help` output from `render_list`
2024-03-08 14:40:06 -07:00
dd90553536 Minor Clean Up of includes and gen_tile.cpp (#402)
_Also_:
* Use `mapnik::freetype_engine::register_fonts` in `gen_tile.cpp` rather than recursing manually
  * Available since `Mapnik` v2 which is no longer supported
2024-03-06 21:18:34 -07:00
5014494191 Fix full Ubuntu map service in docker-compose.yml (#401) 2024-03-05 21:56:10 -07:00
7389b160bf Add renderd_config.c usage into render_* targets (#400)
* Add support for reading `renderd.conf` to `render_expired`
* Add support for reading `renderd.conf` to `render_list`
* Add support for reading `renderd.conf` to `render_old`
* Add support for reading `renderd.conf` to `render_speedtest`
* Move renderd/render_* execution tests to own files
* Reduce variable scope & minor clean up
* Use `stat` rather than `access` to check if file exists
2024-03-04 12:03:16 -07:00
d5534bda27 Adjust sscanf usage in renderd_config.c (#399) 2024-03-03 15:09:46 -07:00
b37cfd9ebb Handle mod_tile not having a valid storage context (causing a segmentation fault) (#398)
This only seems to happen after a delay pool is created (i.e. when `ModTileEnableTileThrottling` is enabled) and is then being manipulated.
2024-03-02 17:44:56 -07:00
2de946dde1 Split up renderd.conf processing by section type (#397) 2024-03-01 20:04:09 -07:00
04a0bf0f09 Fix segmentation fault in modtile.c:delay_allowed (#395) 2024-02-24 19:11:02 -07:00
a0e9681f51 Additional tests (#394)
* Add tests for `ModTileCacheExtendedHostName` and expired tileOld/TileVeryOld
* Add tests for `ModTileEnableTileThrottling`/`ModTileEnableTileThrottlingXForward`
2024-02-22 21:29:10 -07:00
c31d526441 Cleaning up CTest tests (#393)
* Improve service startup "tests"
* Address intermittent download test failure
  * Caused by load exceeding threshold
* Add more missing bad HTTPD config tests
* Allow for newer `libpng` with Docker tests
  * Also test more image/file types
2024-02-20 19:52:57 -07:00
8fea58ac66 Minor adjustments to renderd_config.c (#392)
* Remove some usages of `sscanf`
* Rename `active_slave` to `active_renderd_section_num`
2024-02-18 22:02:12 -07:00
1c9f8b6fb5 Move renderd.conf processing code into new file (#391)
So that it can be easily re-used by other targets.
2024-02-18 11:07:19 -07:00
fdaa5a5750 Move foreground definition into src/g_logger.c (#390)
And setting it to `0`, no longer requiring defining it everywhere except in foreground executables as needed.

_Also_:
* Require `GLib` >= `v2.50`
  * Needed for [g_log_writer_standard_streams](https://docs.gtk.org/glib/func.log_writer_standard_streams.html)
* Set FreeBSD job `BUILD_PARALLEL_LEVEL` & `TEST_PARALLEL_LEVEL` environment variables via `run`
  * Rather than hard-coding them
2024-02-10 13:34:27 -07:00
19facb07eb Add Arch Linux build documentation and Dockerfile (#389)
_Also_:
* Added more tests
* Moved test code & test-related code into `/tests`
* Run Docker build tests in parallel
2024-02-06 16:14:11 -07:00
7811debcfd Clarify render_list documentation (#388)
_Also_:
* Added checks for `min` & `max` `x`, `y` & `z` tile coordinate options in order to:
  * Ensure they are integers
  * Ensure they are positive

Resolves #387
2024-02-06 12:56:15 -07:00
22a13e8863 Add Autotools build Dockerfiles (#386)
To make local Autotools build testing easier.
2024-02-04 17:11:31 -07:00
460efcc532 Coverage improvements (#385)
* Omit includes directory from coverage
* Use MAPNIK_{MAJOR,MINOR,PATCH}_VERSION for ease of reability
* Cleaning up tests
2024-02-03 22:46:53 -07:00
6d00447862 Improve test coverage (#384)
* Report coverage for almost all jobs
  * Some jobs fail and are omitted
* Add more tests to `gen_tile_test.cpp`
  * Also lightly cleaned up formatting
* Clean up cmake/* files
  * So they don't report an empty version
* Remove commented `check_*` lines from `CMakeLists.txt` file
  * These were left over from the initial CMake support implementation
2024-02-02 14:04:15 -07:00
d6309f6316 Address compiler warnings (#376)
* `includes/config.h.in`
  * Wrap contents in `#ifndef CONFIG_H`
* `includes/mod_tile.h`
  * Include `protocol.h`
    * For `XMLCONFIG_MAX`
  * Include `apr_tables.h`
    * For `apr_array_header_t`/`apr_time_t`/`apr_uint64_t`
  * Include `netinet/in.h`
    * For `in6_addr`/`in_addr_t`
* `src/cache_expire.c`
  * Include `netinet/in.h`
    * For `htonl`/`htons`
* `src/gen_tile.cpp`
  * Use `snprintf` instead of `sprintf`
    * To resolve `warning: 'sprintf' is deprecated` in [`macOS` builds](https://github.com/openstreetmap/mod_tile/actions/runs/7590268635/job/20677236621#step:5:579)
* `src/gen_tile_test.cpp`
  * Use `snprintf` instead of `sprintf`
    * To resolve `warning: 'sprintf' is deprecated` in [`macOS` builds](https://github.com/openstreetmap/mod_tile/actions/runs/7590268635/job/20677236621#step:5:579)
* `src/mod_tile.c`
  * Use `snprintf` instead of `sprintf`
    * To resolve `warning: 'sprintf' is deprecated` in [`macOS` builds](https://github.com/openstreetmap/mod_tile/actions/runs/7590268635/job/20677236621#step:5:579)
  * Cast `os_thread` to `(unsigned long)` in `ap_log_rerror` calls format
    * To resolve `warning: format specifies type 'long' but the argument has type 'apr_os_thread_t'`
    * `os_thread` is an alias of `unsigned long`
  * Use `"%" APR_OFF_T_FMT`/`APR_UINT64_T_FMT "..."` to resolve warnings under `macOS`
    * To resolve `warning: format specifies type 'long' but the argument has type 'off_t'`
      * The format/type for `apr_off_t` varies under `macOS`
        * See [here](8e68a77f61/include/apr.h.in (L610-L645))
    * And resolve `warning: format specifies type 'long' but the argument has type 'apr_uint64_t'`
      * The format/type for `apr_uint64_t` varies under `macOS` 
        * See [here](8e68a77f61/include/apr.h.in (L610-L645))
  * Divide by `2.0` rather than `2`
    * To resolve `possible loss of precision` warning
  * Use `%li` rather than `"%" APR_TIME_T_FMT` as format for `maxAge`
    * It is defined as a `long int`
* `src/render_submit_queue.c`
  * Include `string.h`/`strings.h`
    * To resolve `warning: call to undeclared library function 'bzero'`
    * To resolve `warning: call to undeclared library function 'strncpy'`
    * To resolve `warning: call to undeclared library function 'strdup'`
    * To resolve `warning: call to undeclared library function 'strerror'`
    * To resolve `warning: call to undeclared library function 'strchr'`
    * To resolve `warning: call to undeclared library function 'strlen'`
    * To resolve `warning: call to undeclared library function 'memset'`
  * Cast `performance_stats.stat[i].noRendered` to `(float)`
    * To resolve `possible loss of precision` warning
* `src/renderd.c`
  * Use `snprintf` instead of `sprintf`
    * To resolve `warning: 'sprintf' is deprecated` in [`macOS` builds](https://github.com/openstreetmap/mod_tile/actions/runs/7590268635/job/20677236621#step:5:579)
  * Change `const char` to `char` for `ini_fileExtension`, `ini_mimeType`, `ini_outputFormat`
    * To resolve `warning: format specifies type 'char *' but the argument has type 'const char *'`
* `src/request_queue.c`
  * Add `default case` to `switch` statements
    * To resolve:
      * `warning: enumeration values 'queueRender' and 'queueDuplicate' not handled in switch`
      * `warning: enumeration values 'cmdIgnore', 'cmdDone', and 'cmdNotDone' not handled in switch`
* `src/store_file.c`
  * Cast `pthread_self()` to `(unsigned long)`
    * To resolve `warning: format specifies type 'unsigned long' but the argument has type 'pthread_t'`
* No longer need `CFLAGS=-Wno-implicit-function-declaration`
2024-01-30 09:42:38 -07:00
38fb906be7 Mapnik v4.0.0 (latest) supports using std::filesystem with C++17 (#383) 2024-01-30 00:04:54 -07:00
086cc8892e Add option to detect available cores when num_threads=-1 (#380)
When configuring `num_threads=-1` in `renderd.conf` for a `renderd` section, the value will be set to the number of cores on the running system.
2024-01-29 19:30:35 -07:00
6e2fc56888 Improve gen_tile_test test coverage (#382)
* Don't limit Coverage CTest parallel level to 1
* Additional tests
* Re-enable Fedora Rawhide
2024-01-29 16:11:08 -07:00
9408604811 CTest & GitHub Actions Improvements (#381)
* Run tests in parallel in some `GitHub Actions` Workflows
  * Except for `macOS` & `Coverage` jobs
* Update `Node.js 16` actions to `Node.js 20` versions
  * `actions/upload-artifact`
  * `actions/cache`
* Add server `IP:Port` in `CTest` HTTPD logs
* Make `render_speedtest_*` CTest tests require tiles to be downloaded
* Add & use tile download functions for CTest
  * Including `check` & `remove` "tests"
  * To remove much of the redundant code
* Split up HTTPD0/HTTPD1 downloads in CTest
* Remove Amazon Linux 2/CentOS 7 from CI workflow
* Comment out fedora:rawhide, it's not currently working
* Re-enable CentOS Stream tests
2024-01-24 21:45:21 -07:00
4143ab23bd Prepend host to Listen directive in httpd.conf for CTest (#377)
To follow up on #374
2024-01-22 09:43:17 -07:00
a0a1092721 Report coverage for latest Mapnik as well (#375)
_Also_:
* Don't build unneeded `latest` Mapnik `benchmarks`/`demos`/`utilities`
2024-01-19 16:15:19 -07:00
b3c19c13d9 Allow custom host specification with CTest (#374)
* To facilitate running `memcached` tests for all jobs
  * `CentOS`/`Debian`/`Fedora` jobs don't seem to like running `memcached` on `localhost`
  * `FreeBSD` jobs don't seem to like running `memcached` on `0.0.0.0`
2024-01-19 15:39:07 -07:00
8cca561116 Added tile_dir support to httpd.conf directive AddTileConfig (#370)
_Also_:
- Changed `AddTileConfig` parsing directive
  - From `AP_INIT_RAW_ARGS` to `AP_INIT_TAKE_ARGV`
  - Makes code a bit cleaner, e.g.:
    - Checking for first two "arguments"
    - Iterating "arguments"
- Applied Artistic Style Formatting 
  - After merging #346
- Added deprecation notice when using `AddTileMimeConfig`
- Enabled commented `AddTileConfig`/`AddTileMimeConfig` tests
2024-01-11 21:17:53 -07:00
d761f96c0b Added AddTileConfig documentation to example HTTPD config (#372)
_Also_:
- Removed `AddTileMimeConfig` from example `HTTPD` config
  - No longer needed after merging #346
  - Deprecation notice incoming with #370
- Added additional `AddTileConfig`/`AddTileMimeConfig` tests
  - Testing failing download with invalid map name in `httpd.conf`
  - Testing parsing invalid `httpd.conf`, e.g.:
    - `LoadTileConfigFile` without arguments
    - `LoadTileConfigFile` without non-existent file
- Replaced `grep` with `${GREP_EXECUTABLE}` in tests
2024-01-11 19:46:00 -07:00
097f941fad Added tests for ATTRIBUTION & DESCRIPTION (#371) 2024-01-11 16:51:07 -07:00
09e7fa1d43 Allow host:port specification for MemcacheD storage backend (#369)
So that MemcacheD hosts and ports other than `localhost` and `11211` can be used.

_Also_:
- Added `ctest --parallel` execution support
- Added tests for custom `MemcacheD` {host}:{port}
- Added tests for `AddTileConfig`/`AddTileMimeConfig`
2024-01-10 16:33:08 -07:00
a34672865d Allow key=value parameters for AddTileConfig (#346)
AddTileConfig is a shorthand way of adding tile configs without a renderd.conf; if you use mod_tile with tirex, then you will typically not have a renderd.conf and hence rely on AddTileConfig.

AddTileConfig lacks the richness of renderd.conf configuration; you cannot, for example, configure differing max_zooms or file extensions or mime types.

To partly fix this problem, AddTileMimeConfig was introduced which takes one extra argument - misleadingly, not the MIME type, but the file extension, as a third parameter.

This approach is not scalable and difficult to use because of the positional parameters.

This pull request introduces optional key-value pairs for the AddTileConfig directive. You can now write

     AddTileConfig /tile tile mimetype=image/jpeg extension=jpg maxzoom=22
 
to configure your tile layer. Currently, "mimetype", "maxzoom", "minzoom", and "extension" are supported. 

This is totally backwards compatible, as old-style AddTileConfig directives will still work like they did before. 

The AddTileMimeConfig directive still works but it could be deprecated in favour of this new AddTileConfig syntax.

Co-authored-by: Frederik Ramm <ramm@geofabrik.de>
2024-01-10 11:37:12 -07:00
c7ffa83247 Simplifed docker/docker-compose.yml file (#368)
By taking advantage of `YAML anchors`: https://docs.docker.com/compose/compose-file/10-fragments/

_Also_:
- Added `full` service to `docker/docker-compose.yml` (including real osm data and openstreetmap carto)
  - Under `ubuntu-devel-full`
    - I.E. (`cd docker && docker compose up ubuntu-devel-full`)
  - In order to more fully test
2024-01-06 22:23:50 -07:00
d8cd8dea9a Add tests for MemcacheD storage backend (#362)
_Also_:
* Added background `renderd` process testing
* Added `renderd` section without `stats_file` testing
* Added `render_*` via TCP tests
* Added `render_* --verbose`  tests
* Added `render_expired --delete-from` tests
* Added `render_expired --no-progress` tests
* Added `render_expired --touch-from` tests
* Added `PARAMETERIZE_STYPE=language` map configuration testing
* Added `HTCPHOST`/`HOST` map configuration testing
2024-01-06 13:25:49 -07:00