Commit Graph

623 Commits

Author SHA1 Message Date
b37cfd9ebb Handle mod_tile not having a valid storage context (causing a segmentation fault) (#398)
This only seems to happen after a delay pool is created (i.e. when `ModTileEnableTileThrottling` is enabled) and is then being manipulated.
2024-03-02 17:44:56 -07:00
2de946dde1 Split up renderd.conf processing by section type (#397) 2024-03-01 20:04:09 -07:00
04a0bf0f09 Fix segmentation fault in modtile.c:delay_allowed (#395) 2024-02-24 19:11:02 -07:00
a0e9681f51 Additional tests (#394)
* Add tests for `ModTileCacheExtendedHostName` and expired tileOld/TileVeryOld
* Add tests for `ModTileEnableTileThrottling`/`ModTileEnableTileThrottlingXForward`
2024-02-22 21:29:10 -07:00
c31d526441 Cleaning up CTest tests (#393)
* Improve service startup "tests"
* Address intermittent download test failure
  * Caused by load exceeding threshold
* Add more missing bad HTTPD config tests
* Allow for newer `libpng` with Docker tests
  * Also test more image/file types
2024-02-20 19:52:57 -07:00
8fea58ac66 Minor adjustments to renderd_config.c (#392)
* Remove some usages of `sscanf`
* Rename `active_slave` to `active_renderd_section_num`
2024-02-18 22:02:12 -07:00
1c9f8b6fb5 Move renderd.conf processing code into new file (#391)
So that it can be easily re-used by other targets.
2024-02-18 11:07:19 -07:00
fdaa5a5750 Move foreground definition into src/g_logger.c (#390)
And setting it to `0`, no longer requiring defining it everywhere except in foreground executables as needed.

_Also_:
* Require `GLib` >= `v2.50`
  * Needed for [g_log_writer_standard_streams](https://docs.gtk.org/glib/func.log_writer_standard_streams.html)
* Set FreeBSD job `BUILD_PARALLEL_LEVEL` & `TEST_PARALLEL_LEVEL` environment variables via `run`
  * Rather than hard-coding them
2024-02-10 13:34:27 -07:00
19facb07eb Add Arch Linux build documentation and Dockerfile (#389)
_Also_:
* Added more tests
* Moved test code & test-related code into `/tests`
* Run Docker build tests in parallel
2024-02-06 16:14:11 -07:00
7811debcfd Clarify render_list documentation (#388)
_Also_:
* Added checks for `min` & `max` `x`, `y` & `z` tile coordinate options in order to:
  * Ensure they are integers
  * Ensure they are positive

Resolves #387
2024-02-06 12:56:15 -07:00
22a13e8863 Add Autotools build Dockerfiles (#386)
To make local Autotools build testing easier.
2024-02-04 17:11:31 -07:00
460efcc532 Coverage improvements (#385)
* Omit includes directory from coverage
* Use MAPNIK_{MAJOR,MINOR,PATCH}_VERSION for ease of reability
* Cleaning up tests
2024-02-03 22:46:53 -07:00
6d00447862 Improve test coverage (#384)
* Report coverage for almost all jobs
  * Some jobs fail and are omitted
* Add more tests to `gen_tile_test.cpp`
  * Also lightly cleaned up formatting
* Clean up cmake/* files
  * So they don't report an empty version
* Remove commented `check_*` lines from `CMakeLists.txt` file
  * These were left over from the initial CMake support implementation
2024-02-02 14:04:15 -07:00
d6309f6316 Address compiler warnings (#376)
* `includes/config.h.in`
  * Wrap contents in `#ifndef CONFIG_H`
* `includes/mod_tile.h`
  * Include `protocol.h`
    * For `XMLCONFIG_MAX`
  * Include `apr_tables.h`
    * For `apr_array_header_t`/`apr_time_t`/`apr_uint64_t`
  * Include `netinet/in.h`
    * For `in6_addr`/`in_addr_t`
* `src/cache_expire.c`
  * Include `netinet/in.h`
    * For `htonl`/`htons`
* `src/gen_tile.cpp`
  * Use `snprintf` instead of `sprintf`
    * To resolve `warning: 'sprintf' is deprecated` in [`macOS` builds](https://github.com/openstreetmap/mod_tile/actions/runs/7590268635/job/20677236621#step:5:579)
* `src/gen_tile_test.cpp`
  * Use `snprintf` instead of `sprintf`
    * To resolve `warning: 'sprintf' is deprecated` in [`macOS` builds](https://github.com/openstreetmap/mod_tile/actions/runs/7590268635/job/20677236621#step:5:579)
* `src/mod_tile.c`
  * Use `snprintf` instead of `sprintf`
    * To resolve `warning: 'sprintf' is deprecated` in [`macOS` builds](https://github.com/openstreetmap/mod_tile/actions/runs/7590268635/job/20677236621#step:5:579)
  * Cast `os_thread` to `(unsigned long)` in `ap_log_rerror` calls format
    * To resolve `warning: format specifies type 'long' but the argument has type 'apr_os_thread_t'`
    * `os_thread` is an alias of `unsigned long`
  * Use `"%" APR_OFF_T_FMT`/`APR_UINT64_T_FMT "..."` to resolve warnings under `macOS`
    * To resolve `warning: format specifies type 'long' but the argument has type 'off_t'`
      * The format/type for `apr_off_t` varies under `macOS`
        * See [here](8e68a77f61/include/apr.h.in (L610-L645))
    * And resolve `warning: format specifies type 'long' but the argument has type 'apr_uint64_t'`
      * The format/type for `apr_uint64_t` varies under `macOS` 
        * See [here](8e68a77f61/include/apr.h.in (L610-L645))
  * Divide by `2.0` rather than `2`
    * To resolve `possible loss of precision` warning
  * Use `%li` rather than `"%" APR_TIME_T_FMT` as format for `maxAge`
    * It is defined as a `long int`
* `src/render_submit_queue.c`
  * Include `string.h`/`strings.h`
    * To resolve `warning: call to undeclared library function 'bzero'`
    * To resolve `warning: call to undeclared library function 'strncpy'`
    * To resolve `warning: call to undeclared library function 'strdup'`
    * To resolve `warning: call to undeclared library function 'strerror'`
    * To resolve `warning: call to undeclared library function 'strchr'`
    * To resolve `warning: call to undeclared library function 'strlen'`
    * To resolve `warning: call to undeclared library function 'memset'`
  * Cast `performance_stats.stat[i].noRendered` to `(float)`
    * To resolve `possible loss of precision` warning
* `src/renderd.c`
  * Use `snprintf` instead of `sprintf`
    * To resolve `warning: 'sprintf' is deprecated` in [`macOS` builds](https://github.com/openstreetmap/mod_tile/actions/runs/7590268635/job/20677236621#step:5:579)
  * Change `const char` to `char` for `ini_fileExtension`, `ini_mimeType`, `ini_outputFormat`
    * To resolve `warning: format specifies type 'char *' but the argument has type 'const char *'`
* `src/request_queue.c`
  * Add `default case` to `switch` statements
    * To resolve:
      * `warning: enumeration values 'queueRender' and 'queueDuplicate' not handled in switch`
      * `warning: enumeration values 'cmdIgnore', 'cmdDone', and 'cmdNotDone' not handled in switch`
* `src/store_file.c`
  * Cast `pthread_self()` to `(unsigned long)`
    * To resolve `warning: format specifies type 'unsigned long' but the argument has type 'pthread_t'`
* No longer need `CFLAGS=-Wno-implicit-function-declaration`
2024-01-30 09:42:38 -07:00
38fb906be7 Mapnik v4.0.0 (latest) supports using std::filesystem with C++17 (#383) 2024-01-30 00:04:54 -07:00
086cc8892e Add option to detect available cores when num_threads=-1 (#380)
When configuring `num_threads=-1` in `renderd.conf` for a `renderd` section, the value will be set to the number of cores on the running system.
2024-01-29 19:30:35 -07:00
6e2fc56888 Improve gen_tile_test test coverage (#382)
* Don't limit Coverage CTest parallel level to 1
* Additional tests
* Re-enable Fedora Rawhide
2024-01-29 16:11:08 -07:00
9408604811 CTest & GitHub Actions Improvements (#381)
* Run tests in parallel in some `GitHub Actions` Workflows
  * Except for `macOS` & `Coverage` jobs
* Update `Node.js 16` actions to `Node.js 20` versions
  * `actions/upload-artifact`
  * `actions/cache`
* Add server `IP:Port` in `CTest` HTTPD logs
* Make `render_speedtest_*` CTest tests require tiles to be downloaded
* Add & use tile download functions for CTest
  * Including `check` & `remove` "tests"
  * To remove much of the redundant code
* Split up HTTPD0/HTTPD1 downloads in CTest
* Remove Amazon Linux 2/CentOS 7 from CI workflow
* Comment out fedora:rawhide, it's not currently working
* Re-enable CentOS Stream tests
2024-01-24 21:45:21 -07:00
4143ab23bd Prepend host to Listen directive in httpd.conf for CTest (#377)
To follow up on #374
2024-01-22 09:43:17 -07:00
a0a1092721 Report coverage for latest Mapnik as well (#375)
_Also_:
* Don't build unneeded `latest` Mapnik `benchmarks`/`demos`/`utilities`
2024-01-19 16:15:19 -07:00
b3c19c13d9 Allow custom host specification with CTest (#374)
* To facilitate running `memcached` tests for all jobs
  * `CentOS`/`Debian`/`Fedora` jobs don't seem to like running `memcached` on `localhost`
  * `FreeBSD` jobs don't seem to like running `memcached` on `0.0.0.0`
2024-01-19 15:39:07 -07:00
8cca561116 Added tile_dir support to httpd.conf directive AddTileConfig (#370)
_Also_:
- Changed `AddTileConfig` parsing directive
  - From `AP_INIT_RAW_ARGS` to `AP_INIT_TAKE_ARGV`
  - Makes code a bit cleaner, e.g.:
    - Checking for first two "arguments"
    - Iterating "arguments"
- Applied Artistic Style Formatting 
  - After merging #346
- Added deprecation notice when using `AddTileMimeConfig`
- Enabled commented `AddTileConfig`/`AddTileMimeConfig` tests
2024-01-11 21:17:53 -07:00
d761f96c0b Added AddTileConfig documentation to example HTTPD config (#372)
_Also_:
- Removed `AddTileMimeConfig` from example `HTTPD` config
  - No longer needed after merging #346
  - Deprecation notice incoming with #370
- Added additional `AddTileConfig`/`AddTileMimeConfig` tests
  - Testing failing download with invalid map name in `httpd.conf`
  - Testing parsing invalid `httpd.conf`, e.g.:
    - `LoadTileConfigFile` without arguments
    - `LoadTileConfigFile` without non-existent file
- Replaced `grep` with `${GREP_EXECUTABLE}` in tests
2024-01-11 19:46:00 -07:00
097f941fad Added tests for ATTRIBUTION & DESCRIPTION (#371) 2024-01-11 16:51:07 -07:00
09e7fa1d43 Allow host:port specification for MemcacheD storage backend (#369)
So that MemcacheD hosts and ports other than `localhost` and `11211` can be used.

_Also_:
- Added `ctest --parallel` execution support
- Added tests for custom `MemcacheD` {host}:{port}
- Added tests for `AddTileConfig`/`AddTileMimeConfig`
2024-01-10 16:33:08 -07:00
a34672865d Allow key=value parameters for AddTileConfig (#346)
AddTileConfig is a shorthand way of adding tile configs without a renderd.conf; if you use mod_tile with tirex, then you will typically not have a renderd.conf and hence rely on AddTileConfig.

AddTileConfig lacks the richness of renderd.conf configuration; you cannot, for example, configure differing max_zooms or file extensions or mime types.

To partly fix this problem, AddTileMimeConfig was introduced which takes one extra argument - misleadingly, not the MIME type, but the file extension, as a third parameter.

This approach is not scalable and difficult to use because of the positional parameters.

This pull request introduces optional key-value pairs for the AddTileConfig directive. You can now write

     AddTileConfig /tile tile mimetype=image/jpeg extension=jpg maxzoom=22
 
to configure your tile layer. Currently, "mimetype", "maxzoom", "minzoom", and "extension" are supported. 

This is totally backwards compatible, as old-style AddTileConfig directives will still work like they did before. 

The AddTileMimeConfig directive still works but it could be deprecated in favour of this new AddTileConfig syntax.

Co-authored-by: Frederik Ramm <ramm@geofabrik.de>
2024-01-10 11:37:12 -07:00
c7ffa83247 Simplifed docker/docker-compose.yml file (#368)
By taking advantage of `YAML anchors`: https://docs.docker.com/compose/compose-file/10-fragments/

_Also_:
- Added `full` service to `docker/docker-compose.yml` (including real osm data and openstreetmap carto)
  - Under `ubuntu-devel-full`
    - I.E. (`cd docker && docker compose up ubuntu-devel-full`)
  - In order to more fully test
2024-01-06 22:23:50 -07:00
d8cd8dea9a Add tests for MemcacheD storage backend (#362)
_Also_:
* Added background `renderd` process testing
* Added `renderd` section without `stats_file` testing
* Added `render_*` via TCP tests
* Added `render_* --verbose`  tests
* Added `render_expired --delete-from` tests
* Added `render_expired --no-progress` tests
* Added `render_expired --touch-from` tests
* Added `PARAMETERIZE_STYPE=language` map configuration testing
* Added `HTCPHOST`/`HOST` map configuration testing
2024-01-06 13:25:49 -07:00
6c1d89005e Make v3.x.x the minimum required Mapnik version (#365)
Remove support for Mapnik versions < [3.0.0](https://mapnik.org/news/release-3.0.0), which have been around since 2015
2024-01-05 13:15:28 -07:00
ff06794c00 Added missing signal handlers (#364)
`renderd` coverage is otherwise not properly reported

_Also:_
- Removed `freebsd12` from CI
  - Packages are no longer provided (http://pkg.freebsd.org/)
- Added test for `HTCP host name` specification
- Renamed files in `src`/`includes` to match target names
  - `src/daemon.c` → `src/renderd.c`
  - `src/speedtest.cpp` →`src/render_speedtest.cpp`
  - `includes/daemon.h` → `includes/renderd.h`
2024-01-03 19:34:39 -07:00
53b77bc1db Added Docker-based building and testing method (#363)
* Build with Docker
* Use CUSTOM_DEFINES for Mapnik building
* Only run `install-package-and-test.yml` if etc, utils or YAML file have changed
2023-12-30 20:57:44 -07:00
4e8334ed54 Bumped to version 0.7.0 v0.7.0 2023-12-19 20:49:29 -07:00
3c324648e6 Minor coverage improvements and CTest clean up (#361) 2023-12-14 17:31:52 -07:00
4a42757f2e Fix macOS cmake --install command failing due to Read-only file system (#350)
On `macOS`, the directory `/usr` is read-only (except for `/usr/local`, at least).
2023-12-12 15:05:54 -07:00
8e3e8fb34f Add man page for renderd.conf (#357)
* Update AUTHORS file
* Add renderd.conf man to CMakeLists.txt & Makefile.am
* Minor updates to other man pages as well
2023-12-12 13:44:23 -07:00
8abe6eed6d Fix DEFAULT_ATTRIBUTION value and add tests (#360)
The value for `DEFAULT_ATTRIBUTION` in `includes/mod_tile.h` needed to be escaped, it was causing a JSON parsing failure.
* Added basic test for `/.../.../tile-layer.json` to ensure parsing is successful
* Added basic tests for `/metrics` & `/mod_tile` URLs
2023-12-12 12:48:15 -07:00
925183a960 Make /status and /dirty URLs controlable with apache conf (#359)
* Allow the …/status URL to be turned on or off (default of on remains)

Add ModTileEnableStatusURL On/Off to allow the …/status to be turned on
(default) or off. Previous behaviour has …/status on, which is not
changed in this patch.

* Allow the …/dirty URL to be turned on or off (default of on remains)

Add ModTileEnableDirtyURL On/Off to allow the …/dirty to be turned on
(default) or off. Previous behaviour has …/dirty on, which is not
changed in this patch.

* Make …/dirty & …/status disabled by default.

This is a breaking change. To get the old default behaviour back, you
must now manually add `ModTileEnableStatusURL On` &
`ModTileEnableDirtyURL On` to your apache configuration.

* Change default for ModTileEnable Dirty/Status URL to On

* Add log output for when `/dirty`/`/status` is Off

* Add tests for "ModTileEnable{Dirty|Status}URL Off"

---------

Co-authored-by: Amanda McCann <amanda@technomancy.org>
2023-12-11 20:47:50 -07:00
4580b2e887 Improve FreeBSD CI Workflows speeds (#358)
The `freebsd-vagrant-action` action now supports running on `Linux` runners, which almost halves the duration of `FreeBSD` building and testing.
2023-12-06 13:32:55 -07:00
011af8a683 Upgrade Catch2 from v2.13.9 to v2.13.10 (#356)
Keep it up to date, `v3.x.x` requires C++ >= 14, so we'll hold off on that
2023-11-28 12:23:03 -07:00
68e46a2eec Added test for installing from distribution packages (#330)
In order to ensure that the instructions contained within `README.rst` also work when installing `mod_tile` packages from `Debian`/`Ubuntu` repositories, I have added a basic test which first installs those packages and then executes the same commands from the aforementioned instructions.

Here is an example of a workflow run:
https://github.com/hummeltech/mod_tile/actions/runs/5638547864

**Also:**
* Add additional image format types into example configuration instructions in `README.rst`
  * Which are already referenced in [utils/example-map/index.html](https://github.com/openstreetmap/mod_tile/blob/master/utils/example-map/index.html#L28-L39)
* Add step in `README.rst` to ensure `/var/run/renderd` exists
2023-11-27 21:17:25 -07:00
030c35373e Fix build artifacts not uploading (#355) 2023-11-27 20:06:52 -07:00
0b25d54b1e Allow branch names containing CI to run all jobs (#354) 2023-11-26 13:52:03 -07:00
7e92e4c88a Upload astyle formatting patch file as artifact during CI pipeline run (#353) 2023-11-26 13:51:41 -07:00
bc8126cc69 Update GitHub Actions images (#352)
* Remove `macOS 11`
  * Homebrew no longer supports [`macOS < 12`](https://docs.brew.sh/Installation)
* Add `FreeBSD 14`
* Remove `FreeBSD 12`
* Add `Fedora 39`
* Remove `Fedora 37`
* Add `Ubuntu` rolling
* Allow tests using latest `Mapnik` to fail
  * Updates occasionally cause failures to occur
2023-11-26 12:21:09 -07:00
e94fde2982 Add a --no-progress switch to render_expired (#351)
When running `render_expired` as part of regular updates lots of progress messages just clog up the logs.
2023-11-25 18:31:52 -07:00
2577716b0f FreeBSD quarterly repo is now missing mapnik (#348)
Mapnik v3.1.0 requires a patch in order to support Boost v1.8.3.

A fix has been applied, but has not yet made it into the quarterly repository:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=274166
2023-10-28 11:54:48 -07:00
c2896c78ec Added OpenSUSE support to build & test pipeline (#344)
* Also added build documentation

Resolves #89
2023-10-05 15:05:53 -07:00
184f1430f5 CentOS 7 not working with actions/checkout@v4 (#342)
Currently failing:
```
/__e/node20/bin/node: /lib64/libm.so.6: version `GLIBC_2.27' not found (required by /__e/node20/bin/node)
/__e/node20/bin/node: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /__e/node20/bin/node)
/__e/node20/bin/node: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /__e/node20/bin/node)
/__e/node20/bin/node: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /__e/node20/bin/node)
/__e/node20/bin/node: /lib64/libc.so.6: version `GLIBC_2.28' not found (required by /__e/node20/bin/node)
/__e/node20/bin/node: /lib64/libc.so.6: version `GLIBC_2.25' not found (required by /__e/node20/bin/node)
```
2023-10-02 12:52:19 -07:00
bc0ddfef9c Added more linters (cmakelint & prettier) (#341)
For linting `CMakeLists.txt` & `*.cmake` files (`cmakelint`) as well as various other formats such as `HTML`, `MarkDown` & `YAML` (`prettier`.)

### Also:
* Upgrade `actions/checkout` from `v3` to `v4`
* Output coverage summary to GitHub Job step summary
* Minor build docs cleanup for `CentOS`  & `FreeBSD`
2023-10-02 10:17:22 -07:00
f17bb75a3f Allow FreeBSD job to fail (#340)
FreeBSD's `mapnik` package is currently not being built due to some minor errors, once it is fixed, this job will begin to pass again.

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=274166
2023-10-01 08:56:37 -07:00