Commit Graph

3222 Commits

Author SHA1 Message Date
0b3fa706f0 Fix parsing problems in style file reader
Some variables were not initialized correctly when parsing a style file,
which lead to some surprising behaviour with flags of one config line
re-used by the next if the flags field of that line was empty.

This could also have lead to buffer overflows in the first line being
parsed.

This commit also adds some more tests for the parsing code.

See #1590
2021-12-14 15:10:47 +01:00
cff2a2d637 Merge pull request #1583 from joto/fix-logging-while-progress
Fix logging while progress display is on
2021-12-09 10:38:41 +01:00
3dbdd24125 Factor common code out of template function 2021-12-03 21:19:31 +01:00
a7c174428f Fix: Add return character before logging when progress display is shown
When there is an active progress display, log messages would show up
after the progress display instead of the next line.

Fixes #1570
2021-12-03 21:19:19 +01:00
456883663e Use parameter name "args" instead of the strangly named "tags" 2021-11-29 11:50:32 +01:00
136dfa1b84 Merge pull request #1578 from joto/github-ci-win
Github action: Add Windows CI build
2021-11-10 16:02:40 +01:00
682dd55c6c Github action: Add Windows CI build 2021-11-10 14:08:33 +01:00
8c4670dc4f Merge pull request #1574 from chenrui333/patch-1
ci: improve the workflow
2021-11-10 13:29:09 +01:00
ab7659d124 Merge pull request #1579 from lonvia/drop-more-compilers
remove CI runs on Ubuntu 16 images
2021-11-10 13:27:46 +01:00
c31046b522 remove CI runs on Ubuntu 16 images
These images are not available anymore. This means that we won't
be testing gcc < 7 and clang < 8 anymore, so add a note to that
end in the prerequisites section.
2021-11-10 09:00:32 +01:00
rui
67cc640815 ci: update macos job
- be explicit about the LUA_VERSION
- specify `''` for test-wrapper
2021-10-01 18:15:27 -04:00
rui
bd98ade0df ci: improve the workflow 2021-10-01 12:45:38 -04:00
a50c54e2c7 Merge pull request #1567 from joto/fix-memory-release
Clean up pointers to middle_query_t in outputs to release memory
2021-09-05 21:23:54 +02:00
81aa4994a8 Merge pull request #1569 from joto/cleanup-test
Cleanup remnant of the times we allowed multiple outputs
2021-09-04 17:11:39 +02:00
e190190a14 Cleanup remnant of the times we allowed multiple outputs 2021-09-04 14:44:09 +02:00
f2ee699799 Merge pull request #1560 from joto/fix-confusing-log-msg
Fix confusing log message
2021-09-03 10:21:02 +02:00
90f9392806 Use proper accessor for m_mid member in output_t class 2021-09-03 09:45:22 +02:00
58502404cc Clean up pointers to middle_query_t in outputs to release memory
The outputs have a shared pointer to the middle_query_t to access the
data held by the middle. This in turn holds pointers to potentially
large datastructures and the flat node file. We need to release those
pointers after we don't need them any more so that if and when the
middle doesn't need those data structures any more they get cleaned up.

In effect this releases the memory used for the cache and flat node
store earlier.

Fixes #1563
2021-09-03 09:30:50 +02:00
b1aed358a4 Fix confusing log message
The message "Done postprocessing on table '{}' in {}" was logged twice
when `--drop` is used. This changes one of the log messages to the
more specific "Table '{}' dropped in {}".
2021-08-31 21:49:16 +02:00
f8beea1419 Merge pull request #1557 from joto/run-analyze-only-in-create-mode
Run ANALYZE on middle tables only in create mode
2021-08-29 23:01:29 +02:00
605dc26a3f Run ANALYZE on middle tables only in create mode
Fixes #1556
2021-08-24 11:25:15 +02:00
499300c3b2 Merge pull request #1548 from geofabrik/info-command
add a 'status' command which prints the lag (& in JSON)
2021-08-20 10:03:33 +02:00
2d3e6f1f94 Add 'status' command to osm2pgsql-replication
Prints the current replication status, and with --json prints that as
JSON data
2021-08-18 11:49:45 +02:00
717ee5910c Merge pull request #1554 from geofabrik/update-osm2pgsql-replication
Update docs on making `osm2pgsql-replication` man page
2021-08-18 09:16:45 +02:00
26583c2e7c replication: typo fix 2021-08-17 21:45:38 +02:00
0435ee0d2d replication: document that this unneeded section should be removed 2021-08-17 17:46:02 +02:00
fde5f17d2e replication: remove the authors section which shouldn't be checked in 2021-08-17 17:45:51 +02:00
bd7b444000 Merge pull request #1553 from geofabrik/arg-prefix
Add -p (for --prefix) argument for osm2pgsql-replication too
2021-08-12 20:48:46 +02:00
89f8bb6223 regenerate the osm2pgsql-replication manpage now with the command 2021-08-12 17:56:07 +02:00
a03e1fa65a update the docs on how to build the docs, to (now) include the project url in the man page 2021-08-12 17:55:37 +02:00
c8fab14125 update docs on how to build the docs to include the project name 2021-08-12 17:54:35 +02:00
e0b125775b update doc on how to build doc 2021-08-12 17:45:27 +02:00
de9dce01c5 update doc on how to build doc 2021-08-12 17:45:19 +02:00
f04896a3ab Add -p (for --prefix) argument for osm2pgsql-replication too
Regular osm2pgsql takes -p as an alias for --prefix, so make them match
2021-08-12 17:31:45 +02:00
b3a9b732b6 Merge pull request #1546 from geofabrik/support-older-python-versions
support older versions of python with osm2pgsql-replication
2021-08-05 10:57:27 +02:00
d2ed429325 support older versions of python
Older versions don't have .fromisoformat() function
2021-08-04 10:06:02 +02:00
c2b54e2125 prepare release 1.5.1 1.5.1 2021-07-29 09:49:35 +02:00
49c9f9af69 Fix Github issue template 2021-07-25 14:10:31 +02:00
263108d3ae Merge pull request #1544 from joto/issue-template
Add Github issue template for bug reports
2021-07-25 13:55:47 +02:00
cd9022cced Add Github issue template for bug reports 2021-07-25 10:45:03 +02:00
9429db3ed9 Merge pull request #1542 from joto/trigger-func-immutable
Remove IMMUTABLE volatility classification from trigger function
2021-07-20 22:27:22 +02:00
b7067b8f95 Remove IMMUTABLE volatility classification from trigger function
It is not clear how exactly the volatility classification works in
trigger functions, so it is better to be on the safe side here and not
declare the function IMMUTABLE (which makes it the default VOLATILE
instead).

https://www.postgresql.org/docs/current/xfunc-volatility.html
2021-07-20 10:20:50 +02:00
342403ba07 Merge pull request #1541 from joto/fix-node-location-store
Bugfix: Nodes not stored in node location store correctly
2021-07-20 10:15:30 +02:00
284a3e6f8f Bugfix: Nodes not stored in node location store correctly
When importing a planet file or a huge extract, something with more than
about 1 billion nodes, the new RAM node location could overflow a 32bit
"offset" value which meant that the node locations would not be found
again. The result were missing features, because osm2pgsql just ignores
features with geometries that can not be built due to missing node
locations.

This fix has two parts: First, it fixes the typo which made the max
block size too large. This typo lead to a doubling of the max block size
which directly lead to the bug. The other fix is to check the "offset"
value and, if it becomes too large, start a new block. This probably
isn't necessary for "normal" OSM data which should usually work with the
new max block size, but it provides a safety net in case some the block
size becomes too large for some data.

See #1540
2021-07-20 09:23:21 +02:00
75d41a0349 Merge pull request #1534 from joto/fix-typo
Fix typo: disabled
2021-07-09 11:11:04 +02:00
ce282fecab Merge pull request #1531 from joto/input-check-versions
Fix input data check: Two versions of same object are not allowed
2021-07-09 09:18:49 +02:00
663bda3d77 Fix typo: disabled
Fixes #1533
2021-07-09 09:06:34 +02:00
79ecae80a5 Fix input data check: Two versions of same object are not allowed
Two versions of the same object are not allowed in the input data. This
can, for instance, happen when reading un-simplified change files.
2021-07-07 16:03:06 +02:00
fde0a78463 Merge pull request #1529 from joto/expose-config-dir-flex
Expose config dir in osm2pgsql.config_dir Lua var in flex output
2021-07-07 14:42:40 +02:00
c6da7bcc96 Expose config dir in osm2pgsql.config_dir Lua var in flex output
The directory where the Lua config file was read from is now available
in the osm2pgsql.config_dir variable. This is useful, for instance, when
we want to include other files into our config file.
2021-07-07 11:25:39 +02:00