50 Commits

Author SHA1 Message Date
f0811e8f29 Astyle/flawfinder updates (#487)
* Apply astyle updates from newer version of astyle
* Update `github/codeql-action/upload-sarif`
2025-06-01 08:20:43 -07:00
9742572c85 Move xyzo_to_meta into proper header file (#456) 2024-07-02 16:34:12 -07:00
e89046b5ee Also use renderd_config for mod_tile (#440)
Follow-up to #391 & #400
2024-06-19 09:20:50 -07:00
ba0d2af705 Increase XMLCONFIGS_MAX default from 10 to 100 (#434) 2024-06-04 19:42:31 -07:00
743cf651b1 Follow up to #190 for render_list (#417)
* Fixed `render_list` build needs `libm`
* Added tests
* Updated man page
* Created function to set double
* Cleaned up / optimized a bit
* Removed `--dont-render` option
2024-03-21 13:22:02 -07:00
0aa7607d7b Updates to "full" Docker containers (#404)
* To help with testing rendering with real data and styles
2024-03-08 15:38:46 -07:00
dd90553536 Minor Clean Up of includes and gen_tile.cpp (#402)
_Also_:
* Use `mapnik::freetype_engine::register_fonts` in `gen_tile.cpp` rather than recursing manually
  * Available since `Mapnik` v2 which is no longer supported
2024-03-06 21:18:34 -07:00
2de946dde1 Split up renderd.conf processing by section type (#397) 2024-03-01 20:04:09 -07:00
a0e9681f51 Additional tests (#394)
* Add tests for `ModTileCacheExtendedHostName` and expired tileOld/TileVeryOld
* Add tests for `ModTileEnableTileThrottling`/`ModTileEnableTileThrottlingXForward`
2024-02-22 21:29:10 -07:00
8fea58ac66 Minor adjustments to renderd_config.c (#392)
* Remove some usages of `sscanf`
* Rename `active_slave` to `active_renderd_section_num`
2024-02-18 22:02:12 -07:00
1c9f8b6fb5 Move renderd.conf processing code into new file (#391)
So that it can be easily re-used by other targets.
2024-02-18 11:07:19 -07:00
fdaa5a5750 Move foreground definition into src/g_logger.c (#390)
And setting it to `0`, no longer requiring defining it everywhere except in foreground executables as needed.

_Also_:
* Require `GLib` >= `v2.50`
  * Needed for [g_log_writer_standard_streams](https://docs.gtk.org/glib/func.log_writer_standard_streams.html)
* Set FreeBSD job `BUILD_PARALLEL_LEVEL` & `TEST_PARALLEL_LEVEL` environment variables via `run`
  * Rather than hard-coding them
2024-02-10 13:34:27 -07:00
19facb07eb Add Arch Linux build documentation and Dockerfile (#389)
_Also_:
* Added more tests
* Moved test code & test-related code into `/tests`
* Run Docker build tests in parallel
2024-02-06 16:14:11 -07:00
460efcc532 Coverage improvements (#385)
* Omit includes directory from coverage
* Use MAPNIK_{MAJOR,MINOR,PATCH}_VERSION for ease of reability
* Cleaning up tests
2024-02-03 22:46:53 -07:00
d6309f6316 Address compiler warnings (#376)
* `includes/config.h.in`
  * Wrap contents in `#ifndef CONFIG_H`
* `includes/mod_tile.h`
  * Include `protocol.h`
    * For `XMLCONFIG_MAX`
  * Include `apr_tables.h`
    * For `apr_array_header_t`/`apr_time_t`/`apr_uint64_t`
  * Include `netinet/in.h`
    * For `in6_addr`/`in_addr_t`
* `src/cache_expire.c`
  * Include `netinet/in.h`
    * For `htonl`/`htons`
* `src/gen_tile.cpp`
  * Use `snprintf` instead of `sprintf`
    * To resolve `warning: 'sprintf' is deprecated` in [`macOS` builds](https://github.com/openstreetmap/mod_tile/actions/runs/7590268635/job/20677236621#step:5:579)
* `src/gen_tile_test.cpp`
  * Use `snprintf` instead of `sprintf`
    * To resolve `warning: 'sprintf' is deprecated` in [`macOS` builds](https://github.com/openstreetmap/mod_tile/actions/runs/7590268635/job/20677236621#step:5:579)
* `src/mod_tile.c`
  * Use `snprintf` instead of `sprintf`
    * To resolve `warning: 'sprintf' is deprecated` in [`macOS` builds](https://github.com/openstreetmap/mod_tile/actions/runs/7590268635/job/20677236621#step:5:579)
  * Cast `os_thread` to `(unsigned long)` in `ap_log_rerror` calls format
    * To resolve `warning: format specifies type 'long' but the argument has type 'apr_os_thread_t'`
    * `os_thread` is an alias of `unsigned long`
  * Use `"%" APR_OFF_T_FMT`/`APR_UINT64_T_FMT "..."` to resolve warnings under `macOS`
    * To resolve `warning: format specifies type 'long' but the argument has type 'off_t'`
      * The format/type for `apr_off_t` varies under `macOS`
        * See [here](8e68a77f61/include/apr.h.in (L610-L645))
    * And resolve `warning: format specifies type 'long' but the argument has type 'apr_uint64_t'`
      * The format/type for `apr_uint64_t` varies under `macOS` 
        * See [here](8e68a77f61/include/apr.h.in (L610-L645))
  * Divide by `2.0` rather than `2`
    * To resolve `possible loss of precision` warning
  * Use `%li` rather than `"%" APR_TIME_T_FMT` as format for `maxAge`
    * It is defined as a `long int`
* `src/render_submit_queue.c`
  * Include `string.h`/`strings.h`
    * To resolve `warning: call to undeclared library function 'bzero'`
    * To resolve `warning: call to undeclared library function 'strncpy'`
    * To resolve `warning: call to undeclared library function 'strdup'`
    * To resolve `warning: call to undeclared library function 'strerror'`
    * To resolve `warning: call to undeclared library function 'strchr'`
    * To resolve `warning: call to undeclared library function 'strlen'`
    * To resolve `warning: call to undeclared library function 'memset'`
  * Cast `performance_stats.stat[i].noRendered` to `(float)`
    * To resolve `possible loss of precision` warning
* `src/renderd.c`
  * Use `snprintf` instead of `sprintf`
    * To resolve `warning: 'sprintf' is deprecated` in [`macOS` builds](https://github.com/openstreetmap/mod_tile/actions/runs/7590268635/job/20677236621#step:5:579)
  * Change `const char` to `char` for `ini_fileExtension`, `ini_mimeType`, `ini_outputFormat`
    * To resolve `warning: format specifies type 'char *' but the argument has type 'const char *'`
* `src/request_queue.c`
  * Add `default case` to `switch` statements
    * To resolve:
      * `warning: enumeration values 'queueRender' and 'queueDuplicate' not handled in switch`
      * `warning: enumeration values 'cmdIgnore', 'cmdDone', and 'cmdNotDone' not handled in switch`
* `src/store_file.c`
  * Cast `pthread_self()` to `(unsigned long)`
    * To resolve `warning: format specifies type 'unsigned long' but the argument has type 'pthread_t'`
* No longer need `CFLAGS=-Wno-implicit-function-declaration`
2024-01-30 09:42:38 -07:00
ff06794c00 Added missing signal handlers (#364)
`renderd` coverage is otherwise not properly reported

_Also:_
- Removed `freebsd12` from CI
  - Packages are no longer provided (http://pkg.freebsd.org/)
- Added test for `HTCP host name` specification
- Renamed files in `src`/`includes` to match target names
  - `src/daemon.c` → `src/renderd.c`
  - `src/speedtest.cpp` →`src/render_speedtest.cpp`
  - `includes/daemon.h` → `includes/renderd.h`
2024-01-03 19:34:39 -07:00
4e8334ed54 Bumped to version 0.7.0 2023-12-19 20:49:29 -07:00
8abe6eed6d Fix DEFAULT_ATTRIBUTION value and add tests (#360)
The value for `DEFAULT_ATTRIBUTION` in `includes/mod_tile.h` needed to be escaped, it was causing a JSON parsing failure.
* Added basic test for `/.../.../tile-layer.json` to ensure parsing is successful
* Added basic tests for `/metrics` & `/mod_tile` URLs
2023-12-12 12:48:15 -07:00
925183a960 Make /status and /dirty URLs controlable with apache conf (#359)
* Allow the …/status URL to be turned on or off (default of on remains)

Add ModTileEnableStatusURL On/Off to allow the …/status to be turned on
(default) or off. Previous behaviour has …/status on, which is not
changed in this patch.

* Allow the …/dirty URL to be turned on or off (default of on remains)

Add ModTileEnableDirtyURL On/Off to allow the …/dirty to be turned on
(default) or off. Previous behaviour has …/dirty on, which is not
changed in this patch.

* Make …/dirty & …/status disabled by default.

This is a breaking change. To get the old default behaviour back, you
must now manually add `ModTileEnableStatusURL On` &
`ModTileEnableDirtyURL On` to your apache configuration.

* Change default for ModTileEnable Dirty/Status URL to On

* Add log output for when `/dirty`/`/status` is Off

* Add tests for "ModTileEnable{Dirty|Status}URL Off"

---------

Co-authored-by: Amanda McCann <amanda@technomancy.org>
2023-12-11 20:47:50 -07:00
011af8a683 Upgrade Catch2 from v2.13.9 to v2.13.10 (#356)
Keep it up to date, `v3.x.x` requires C++ >= 14, so we'll hold off on that
2023-11-28 12:23:03 -07:00
fe238926d4 Increase queue limits to reflect modern machines (#336) 2023-09-27 08:17:56 +02:00
f652ab1521 Configurable ImageIO output format (#318)
* Added the possibility to provide ImageIO's output format in the TYPE string.
For instance:
TYPE=png image/png
TYPE=png image/png png256
TYPE=png image/png png24
TYPE=png image/png png

* Removed useless comments.
Added documentation for output format.

* Added tests for 'webp' output format

* Also added tests for 'png32' output format

* Fix 'webp' sums vary based on version

* Add JPG & use multiple layers on example-map

---------

Co-authored-by: Tim <autintim@gmail.com>
2023-07-20 23:41:18 +02:00
6e20fba5e9 Generate config files and directories with CMake (#316)
* Generate config files and directories with CMake

CMake >= 3.13 required due to 'https://cmake.org/Bug/view.php?id=14444'

* Update CI pipeline
2023-07-20 23:34:21 +02:00
f4ee376b35 Build mod_tile with cmake 2023-03-01 11:57:10 +00:00
2325f84d1b Upgrade catch.hpp to Catch2 v2.13.9 2022-07-24 18:30:51 +00:00
296f888752 Resolve various compiler warnings and errors (#279)
* Resolved warnings in `src/daemon.c`

I.E.:
```
src/daemon.c:835:16: warning: initialization discards 'const' qualifier from pointer target type [-Wdiscarded-qualifiers]
   char *name = iniparser_getsecname(ini, section);
                ^~~~~~~~~~~~~~~~~~~~
src/daemon.c:860:20: warning: initialization discards 'const' qualifier from pointer target type [-Wdiscarded-qualifiers]
    char *ini_uri = iniparser_getstring(ini, buffer, (char *)"");
                    ^~~~~~~~~~~~~~~~~~~
```

* Resolved warning in `src/mod_tile.c`

I.E.:
```
./src/mod_tile.c:327:53: warning: format '%d' expects argument of type 'int', but argument 9 has type 'long unsigned int' [-Wformat=]
      ap_log_rerror(APLOG_MARK, APLOG_WARNING, 0, r, "request_tile: Failed to read response from rendering socket. Got %d bytes but expected %d. Errno %d (%s)",
                                                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
             ret, sizeof(struct protocol_v2), errno, strerror(errno));
                  ~~~~~~~~~~~~~~~~~~~~~~~~~~
                  |
                  long unsigned int
```

* Resolved `cppcheck`-detected error

I.E.:
```
src/store_rados.c:355:3: error: Memory leak: store [memleak]
  return NULL;
  ^
```

* Fix `Does not check for buffer overflows (CWE-120)` in `src/daemon.c`

* Fix `Does not check for buffer overflows (CWE-120)` in `src/cache_expire.c`

* Resolved `cppcheck`-detected error

I.E.:
```
src/parameterize_style.cpp:43:3: error: Memory leak: data [memleak]
  return;        //No parameterization given
  ^

* url is `char *`, unlike buffer, size is not known
2022-05-29 18:29:21 +00:00
42e4f0d283 Added PID file path configurability (#282)
* Added PID file path configurability
2022-05-26 20:36:54 +00:00
f0453c37d3 Allow connections over TCP besides Unix sockets
Signed-off-by: Stephan Austermühle <au@hcsd.de>
2022-01-06 20:32:50 +00:00
032d90d130 Reverted change of copyright owners on included source files 2021-08-17 14:43:40 +00:00
a14924a999 Logging Improvements
* Using [`GLib Logging Framework`](https://developer.gnome.org/programming-guidelines/stable/logging.html.en) for logging
  * Created new `g_logger` function for logging
    * Outputs to `stdout`/`stderr` only when running in `foreground`
      * `stderr` for `message`, `warning`, `critical` & `error` levels
      * `stdout` for `debug` & `info` levels
        * Use `G_MESSAGES_DEBUG=all` environment to enable `debug` to print
    * Otherwise, output will be to `syslog` or `systemd journal` (when appropriate)
* Standardized usage of `{LOG_PRIORITY}: ` prefix in log messages
  * Only when using `syslog`, otherwise `GLib Logging` will take care of it
* Changed `fprintf(stderr`, `printf` & `perror` calls to `g_logger` calls
  * You might want to check them out closely to make sure I chose the right levels
  * No changes to `logging/output` were made to "`foreground`" programs (I.E. `render_*`)
* Changed `0`,`1` to `no_argument`,`required_argument` in `getopt_long`'s `long_options`
  * Fixed `renderd`'s `foreground` opt (should be `no_argument` [0] rather than `reguired_argument` [1])
* Basic test for `mod_tile` module
* ~~Extended `renderd` log priority onto Mapnik's logger~~
2021-07-28 14:46:50 +00:00
648913dec9 Formatted code as per Linux style 2021-02-15 19:20:30 +01:00
d706d96020 Clarify copyright holders 2020-10-09 15:20:37 +00:00
889a45b5e0 Use /var/cache instead of /var/lib for tiles 2020-10-09 07:52:15 +00:00
8d711039c8 Merge pull request #213 from xamanu/run-for-deamon
Adhere to FHS 3.0 and use daemon under /run.
2020-09-24 11:24:05 +01:00
de41cb1392 Adhere to FHS 3.0 and use daemon under /run. 2020-09-03 19:32:31 +00:00
a22065b8ae Move variables to prevent duplicates 2020-05-23 12:42:07 +01:00
1d43b86762 Merge branch 'master' of github.com:openstreetmap/mod_tile 2014-08-13 08:11:35 -07:00
7e9abfbb29 SCALE option to produce retina tiles 2014-07-09 18:23:41 +04:00
122adae7fb osx compile fixes 2014-05-23 14:09:13 -07:00
13da3afe70 render_list was not blocking on requests and would swamp renderd with requests 2013-12-24 01:03:20 +01:00
66e5538dc4 fix gcc errors 2013-12-05 17:47:04 +01:00
5dffe058e6 fix gcc errors 2013-12-05 16:35:49 +01:00
99357a53ab Improve backwards and forward compatibility for renderd protocol v3/v2 2013-10-20 16:03:32 -06:00
c4aabc5701 Enable mapnik style parameterization
Provide a parameterization function for internationalizing the names
2013-10-13 23:26:31 -06:00
e70bfafa20 Extend the storage backends to support options 2013-10-13 23:25:14 -06:00
5941e23ced Support options for tile layers in mod_tile.
If a tile layer has options enabled, the URL changes to

http://localhost/style/options/z/x/y.ext
2013-10-13 23:23:10 -06:00
fdc317a8f3 Update mod_tile to renderd protocol to version 3.
Include an "options" string and a "mimetype" string.
2013-10-13 23:22:08 -06:00
dd57fa7ea3 significantly increase the load thresholds after which mod_tile reduces rendering
sets the new MaxLoadMissing to 50 and MaxLoadOld to 16.

With the prevalence of multi core servers today, the previous
default thresholds of 10 and 5 were becoming to lower than the average
CPU count. Furthermore, load is not an ideal measure, as it includes I/O
load and much of the I/O load might well come from serving rather than
rendering. Especially if the database is on a separate SSD. Lastly, using
the thresholds in mod_tile isn't super helpful, as due to hysteresis of
 rendering queues stop submitting tiles at a time of high load is not
particularly helpful without stopping rendering from the queue.
2013-08-10 23:09:54 -06:00
1b4c56d8ed Add another priority level (RenderLow) to accommodate rerenders after style changes
In addition to Render and RenderPrio, add another priority level of RenderLow.

The idea is that if you rerender all tiles by touching planet-import-complete and
thereby likely overloading the server's rendering capacity you want to prioritize rerendering
of tiles that were explicitly marked dirty due to data changes over rerendering
due to planet-import-complete updates.

This is achieved by splitting rendering requests by if they are "old", or "very old".
As the tile expiry sets back the modification date many years, mod_tile splits
rendering requests based on how old the tiles are. If they are no more than a
certain threshold (currently set to one year) old, they are rendered with lower priority.
2013-08-10 23:05:28 -06:00
273973ed03 Clean up mod_tile directory by moving source files into a common src/ sub directory
also move includes into includes/
2013-08-09 18:46:10 -06:00