3432 Commits

Author SHA1 Message Date
f66c11f25c Support retrying on 504 responses 2024-11-15 21:53:38 +01:00
c91afc0d50 Add test for 504 responses 2024-11-15 21:53:38 +01:00
ff735fce14 * libwget/iri.c (wget_iri_parse): Check for port number validity 2024-11-01 18:22:41 +01:00
1d52d85908 * configure.ac: Avoid -Wc11-c2x-compat (issue in config.h) 2024-11-01 13:16:17 +01:00
450f0056c5 * README.md: Amend license section (fix syntax-check) 2024-11-01 10:55:32 +01:00
f6fe8a2d08 * src/wget.c: Fix safe_write, safe_read return types 2024-11-01 10:38:57 +01:00
c952ae1e6b * configure.ac: Suppress several warnings coming with gnulib 01.11.2024 2024-11-01 10:37:26 +01:00
1d6632a31c * libwget/ssl_wolfssl.c (ShowX509): Add semicolon to end of XFREE()
Fixes https://github.com/rockdaboot/wget2/issues/351
2024-11-01 10:12:59 +01:00
92b52380e3 Update gnulib
* gnulib: Update git module.
* bootstrap: Update from gnulib.
2024-11-01 09:47:30 +01:00
c19ea83788 * src/wget.c (program_deinit): Minor cleanup 2024-10-31 19:45:15 +01:00
c7ed8afc08 * src/wget.c: Fix parameter names in function prototypes 2024-10-27 18:31:39 +01:00
915178ab61 Wake the main thread on downloader shutdown
* src/wget.c (downloader_thread): Signal main() on exit.

Without this, the main thread may get stuck waiting for signals from
downloaders while those have already exited and will never signal it.
Observed with --wait.

Fixes: https://github.com/rockdaboot/wget2/issues/348

Copyright-paperwork-exempt: Yes
2024-10-03 18:55:46 +02:00
2c936eb8f0 * src/wget.c: Wait for progress thread during test runs 2024-10-03 18:04:44 +02:00
253d8daf41 * tests/valgrind-suppressions: Suppress libc6 2.40 getaddrinfo/memmove 2024-10-03 18:03:41 +02:00
dc8966d906 Don't log URI userinfo to logs
* include/wget/wget.h (struct wget_iri_st): Add safe_uri member.
* libwget/iri.c: New function create_safe_uri(),
  (wget_iri_unescape_url_inline): Check iri->userinfo,
  (wget_iri_parse): Likewise,
  (wget_iri_clone): Likewise,
  (wget_iri_relative_to_abs): Remove debug logs,
  (wget_iri_set_scheme): Check iri->userinfo.
* src/blacklist.c: Log safe_uri instead of uri.
* src/host.c: Likewise.
* src/options.c: Likewise.
* src/stats_site.c: Likewise.
* src/wget.c: Likewise,
  (queue_url_from_remote): Move redirection check further down.
* unit-tests/test.c (test_iri_parse): Add checks for safe_uri,
  (test_iri_parse): Add safe_uri to IRI comparison,
  Print safe_uri and password.

Co-authored-by: Tim Rühsen <tim.ruehsen@gmx.de>
2024-10-03 17:25:38 +02:00
8877050c3f Count 0 as an error for SSL_read and SSL_write, per documentation
* libwget/ssl_openssl.c (ssl_transfer): Take 0 as error,
  slightly refactor code.

Fixes https://github.com/rockdaboot/wget2/issues/342

Reported-by: Tobias Brick (he/him) <tobiasb@microsoft.com>
Co-authored-by: Tobias Brick (he/him) <tobiasb@microsoft.com>
2024-09-22 12:12:46 +02:00
182a4c8c0d Merge pull request #345 from tobiasb-ms/tobiasb-ms/ssl-binary-request-body-debug-output
set debug_skip_body for OCSP requests in openssl tls provider
2024-09-22 11:47:14 +02:00
2f1972783e Fix error message for unknown --compression types
* src/options.c (parse_compression): Fix error messages.

Fixes https://github.com/rockdaboot/wget2/issues/346
2024-09-22 11:24:09 +02:00
ffa79a561f Add empty line after header with --save-headers
* libwget/http.c (wget_http_get_response_cb): Remove adding empty line.
* src/wget.c (prepare_file): Add empty line to headers.

Fixes https://github.com/rockdaboot/wget2/issues/347
2024-09-21 20:20:25 +02:00
3359e3e182 set debug_skip_body for OCSP requests in openssl tls provider 2024-09-16 15:34:19 +00:00
ec27488fea Fix downloading multiple files via HTTP/2
* include/wget/wget.h: New function declaration wget_http_connection_receive_only().
* libwget/http.c: New function definition wget_http_connection_receive_only().
* libwget/http.h (struct wget_http_connection_st): Add member goaway.
* libwget/http2.c (struct http2_stream_context): Add member conn,
  (on_frame_recv_callback): Handle NGHTTP2_GOAWAY,
  (wget_http2_send_request): Initialize conn member.
* src/host.c (_release_job): Don't release parts if in 'done' state.
* src/wget.c (process_response_header): Remove handling of LINK headers,
  (process_response): Add handling of LINK headers,
  (downloader_thread): Handle pending responses correctly.

This patch fixes two related issues:
1. With HTTP/2, servers sometimes limit the number of requests per connection.
   Beforethis patch, the connection was closed too early after GOAWAY and pending
   responses weren't received.
2. In _release_job(), already downloaded parts of a file were erroneous released.
   This caused successful metalink and chunked downloads to fail in combination with
   a GOAWAY or a remotely closed connection.
2024-09-11 17:45:09 +02:00
c4c43209d5 Update progress report line every second
* src/wget.c: Add new function progress_report(),
  add new declaration progress_report().
* src/wget.c (main): Start thread progress_report().
* src/bar.c: Increase BAR_THREAD_SLEEP_DURATION from 125 to 1000 millseconds.
* libwget/bar.c (bar_update_slot): Limit ratio to 100%, add TODO.
2024-09-06 16:42:12 +02:00
0c0c108104 Do not overwrite wget-log for 2+ instances
* src/wget.c: Add new function create_unique(),
  (fork_to_background): Make use of create_unique().

Fixes https://github.com/rockdaboot/wget2/issues/330
2024-09-05 19:43:46 +02:00
55a4c145c8 Fix redirections with --no-parent
* src/wget.c: Add new functions add_parent(), is_parent(), matches_parent().
* src/wget.c (queue_url_from_remote): Add redirected URL to parents if
  origin is a parent.
* src/wget.c (queue_url_from_remote): Do not apply robots.txt to redirected
  URL if origin was requested by user.

Fixes https://github.com/rockdaboot/wget2/issues/331
2024-09-05 17:01:25 +02:00
f5344eb415 Support connecting with HTTP/1.0 proxies
* libwget/http.c (establish_proxy_connect): Allow HTTP/1.0 responses

Fixes https://gitlab.com/gnuwget/wget2/-/issues/666
2024-09-05 13:28:32 +02:00
fa638f597c Ignore 1xx HTTP responses
* libwget/http.c (wget_http_get_response_cb): Ignore 1xx responses.

Fixes https://github.com/rockdaboot/wget2/issues/340
2024-09-04 20:30:59 +02:00
a819d0c696 Do not spell-check NEWS and bootstrap.conf 2024-09-04 20:17:58 +02:00
2b1f266ca6 Fix status 8 for failed redirection of robots.txt
* src/wget.c (queue_url_from_remote): Copy job->robotstxt flag for redirections"
* tests/test-redirection.c: Add test for correct status code after robots redirection fails"

Fixes https://gitlab.com/gnuwget/wget2/-/issues/677
2024-08-29 17:00:29 +02:00
7aa492f822 Fix confusing error message on Win32 "No CAs were found in ..."
* libwget/ssl.c (wget_ssl_default_ca_bundle_path): Check if file exists
2024-08-16 23:16:16 +02:00
85e377c002 * src/wget.c (program_init): Move console and signal handler setup up 2024-08-10 14:35:15 +02:00
8a739f0a71 Fix buffer overflow in progress bar
* libwget/bar.c (bar_set_progress): Check for minimum bar size.
2024-07-21 18:46:52 +02:00
29b3fc9db6 Consider connect timeout only when set
* libwget/net.c (set_socket_options): Skip SO_SNDTIMEO if connect timeout is <= 0.

Copyright-paperwork-exempt: Yes
2024-07-14 13:41:02 +02:00
21f41932af Fix ignoring connect timeout (regression)
* libwget/net.c (set_socket_options): Set SO_SNDTIMEO for socket

Reported-by: https://gitlab.com/plvalerio
Co-authored-by: https://gitlab.com/RogerMarcoHernandez
2024-07-07 20:12:17 +02:00
3927727725 * contrib/Dockerfile.win32.static: Remove brotli 2024-07-07 19:35:15 +02:00
7a945d31ae Disable TCP Fast Open by default
* docs/wget2.md: Amended description of --tcp-fastopen.
* src/options.c (struct config config): Disabled TFO.
2024-06-30 19:33:23 +02:00
0e60ece89a Connect using synchronous socket
* libwget/dns.c (getaddrinfo_merging, resolve): Code cleanup.
* libwget/net.c (wget_tcp_connect): Connect first, then set socket to async.

Connecting asynchronously prevented wget2 to fallback to the next IP address
of a host when a connect error occurred.

Reported at https://github.com/rockdaboot/wget2/issues/325
2024-06-30 18:55:38 +02:00
ff881ed201 Fix --dns-cache-preload for IPv6
* src/options.c (preload_dns_cache): Fix parsing IPv6 addresses
2024-06-29 14:40:03 +02:00
0710dda725 * src/wget.c (main): Disable progress bar for non-verbose and debug mode 2024-06-01 18:13:48 +02:00
9b901cbeda * src/bar.c (bar_init): Write info text above progress bar 2024-06-01 18:13:03 +02:00
4b2692120b * libwget/bar.c: Add new function wget_bar_write_line_ext() 2024-06-01 18:11:50 +02:00
a9feeb4979 * src/wget.c (main): Do not switch off info text when progress is enabled 2024-06-01 18:10:21 +02:00
7da3819092 * fuzz/Makefile.am (oss-fuzz): Add -lgmp 2024-05-24 18:26:19 +02:00
25cc1bfbec .gitlab-ci.yml (pages): Install pandoc, doxygen, texinfo 2024-05-21 18:57:02 +02:00
6c0eb48f98 * .gitlab-ci.yml (pages): Explicitly clean docs (required to build HTML) 2024-05-21 16:56:16 +02:00
722f725c47 * .gitlab-ci.yml (pages): Explicitly create HTML docs 2024-05-21 16:27:44 +02:00
98b52d946a * .gitlab-ci.yml (pages): Use make-coverage-badge from git source tree 2024-05-21 14:28:56 +02:00
22a00fae27 * .gitlab-ci.yml: Restructuring 2024-05-21 14:17:20 +02:00
769dc861fc * tests/valgrind-suppressions: Drop libgpgme version from suppression 2024-05-21 13:56:10 +02:00
570a022fd2 * tests/Makefile.am: Add suppression files 2024-05-21 13:56:02 +02:00
4d1504aa53 * fuzz/Makefile.am: Add valgrind-suppressions to EXTRA_DIST 2024-05-21 13:55:50 +02:00