Commit Graph

213 Commits

Author SHA1 Message Date
c8bf9a7112 Fix memleaks 2024-01-03 21:39:10 +01:00
b6cb86f859 Implement --http3-only 2023-12-26 20:20:07 +01:00
0f7ead6187 tmp: Call HTTP/3 functions from wget2 2023-12-26 18:50:32 +01:00
1bab5454f9 Fix IPv6 address representation
* libwget/dns.c (resolve): Fix IPv6 in debug message,
  (wget_dns_cache_ip): Fix IPv6 in error message,
  (wget_dns_resolve): Fix IPv6 in debug message.
* libwget/dns_cache.c (wget_dns_cache_get): Fix IPv6 in debug message.
* libwget/hsts.c (hsts_db_add_entry): Fix IPv6 in debug message.
* libwget/http.c (wget_http_create_request): Fix IPv6 in Host header,
  (establish_proxy_connect): Fix IPv6 in CONNECT and in Host header.
* libwget/ip.c (wget_ip_is_family): Cosmetic change.
* libwget/iri.c (wget_iri_get_connection_part): Fix IPv6 in URL creation.
* libwget/net.c (debug_addr): Fix IPv6 in debug message.
* src/options.c (stats_callback_dns): Fix IPv6 in output.
* src/wget.c (test_modify_hsts): Fix IPv6 in console output,
  (http_create_request): Fix IPv6 in Referer header.
* unit-tests/test.c (test_iri_parse): Add two IPv6 parsing tests.
* libwget/Makefile.am (libwget_dnscache.la): Link with libwget_ip.la,
  (libwget_hsts.la): Link with libwget_ip.la.
2023-12-17 18:52:27 +01:00
8271687e29 Rename internal functions 2023-11-18 18:45:35 +00:00
b23c934422 Rename internal functions 2023-11-18 18:45:35 +00:00
a6ab8a1d0c Remove empty guard 2023-11-18 18:45:35 +00:00
829528217c Extract common HTTP functions
* libwget/http.h: new functions wget_http_connection_is_aborted,
   wget_http_fix_broken_server_encoding,
   wget_decompress_get_body_cb, wget_decompress_error_handler_cb,
   wget_http2_open, wget_http2_close, wget_http2_send_request,
   wget_http2_get_response_cb
 * libwget/http2.c: new file
2023-11-18 18:45:35 +00:00
1a886595e6 Add libproxy support
Add support for libproxy, which is capable to extract desktop
environment proxy configurations from dozens of systems and platforms.
This also enables wget to handle pac/wpad proxy server.

* configure.ac: Add check for libbproxy.
* libwget/http.c (wget_http_open): Retrieve proxy via libproxy.

Copyright-paperwork-exempt: Yes
2023-11-04 16:02:45 +00:00
c569c7f4f8 * libwget/http.c: Fix NULLABLE issues 2023-10-03 18:03:00 +02:00
15ba1db248 Update copyrights 2023-08-31 12:57:30 +02:00
33a6454aa7 gcc specific C constructor/destructors support for MSVC
* libwget/private.h: Define INITIALIZER to support constructors with MSVC.
* libwget/dns.c: Use INITIALIZER for library constructor.
* libwget/http.c: Likewise.
* libwget/init.c: Likewise.
* libwget/random.c: Likewise.
* libwget/ssl_gnutls.c: Likewise.
* libwget/ssl_openssl.c: Likewise.
* libwget/ssl_wolfssl.c: Likewise.

Co-authored-by: Tim Rühsen <tim.ruehsen@gmx.e
2023-08-18 20:40:02 +02:00
66fe329eaa * libwget/http.c (wget_http_send_request): Remove unused variables 2023-08-03 14:48:35 +02:00
c20affea04 * libwget/http.c: Replace VLA in http2 code 2023-08-03 13:57:27 +02:00
5270a2b3a8 * libwget/http.c: Replace VLA 2023-07-31 18:19:18 +02:00
0f1abaa11c Fix alternate port requests to proxies
* include/wget/wget.h (wget_http_request_to_buffer): Add param 'port'.
* libwget/http.c (wget_http_request_to_buffer): Likewise,
  (wget_http_send_request): Call wget_http_request_to_buffer with
  connection port.

Copyright-paperwork-exempt: Yes
2023-07-09 12:28:39 +02:00
279901855e Fix code to silence sc_gettext_printf
* libwget/http.c: Add _() around error messages where missing.
* libwget/net.c: Likewise.
* libwget/ssl_gnutls.c: Likewise.
* libwget/ssl_openssl.c: Likewise.
* src/wget.c: Likewise.
2023-07-08 13:00:40 +02:00
68a11b7287 Add CIDR support in 'no_proxy' for IPv6
* include/wget/wget.h: Fix http_get_no_proxy -> wget_http_get_no_proxy.
* libwget/http.c: New function cidr_v6_match(),
  check for ipv6 CIDRs in wget_http_match_no_proxy().
* unit-tests/test.c (test_match_no_proxy): Add IPv6 CIDR tests.
2023-06-17 23:50:53 +02:00
60d2008d1c * libwget/http.c (parse_no_proxies): Skip leading whitespace 2023-05-22 17:46:02 +00:00
6ab47257f0 CIDR support added for no_proxy
* libwget/http.c: Add function wget_http_cidr_match to support
  CIDR IP method for no_proxy
* unit-tests/test.c: Add testcases to verify the functionality
  of the above function

Fixes #615
2023-05-22 17:46:02 +00:00
1fd5c15d5c Use CONNECT for https_proxy
* libwget/http.c (establish_proxy_connect): New function,
  (wget_http_open): Use CONNECT method for HTTPS proxying.

Fixes #616
2022-11-19 18:10:37 +01:00
64234eeef2 * libwget/http.c (wget_http_get_response_cb): Extend error messages with URL 2022-02-27 18:12:03 +01:00
3dc7f71098 Update copyright year 2022-02-25 17:46:43 +01:00
1c84c4127b Fix obscure Windows (Win11) recv() issue (returns 0 without a reason)
* libwget/http.c (wget_http_get_response_cb): Continue HTTP/2 loop if recv()
  return 0.

Fixes https://github.com/rockdaboot/wget2/issues/256
2022-01-21 17:57:46 +01:00
849b54eeed * libwget/http.c (wget_http_get_response_cb): Amend debug output [skip ci] 2022-01-19 18:41:33 +01:00
d6e5641980 * libwget/http.c (wget_http_response_cb): Fix chunked transfer 2021-07-07 18:19:09 +00:00
b084c177c9 * libwget/http.c (wget_http_get_response_cb): Remove superfluous space in debug msg 2021-01-31 19:21:18 +01:00
c349a7eda7 * libwget/http.c (wget_http_get_response_cb): Fix NULL ptr access 2021-01-24 14:06:07 +01:00
e4f4052ddd * libwget/http.c (wget_http_response): Fix buffer overflow 2021-01-24 13:49:37 +01:00
22162f82b2 Update copyright year 2021-01-22 21:58:38 +01:00
8145e07361 Fix findings from scan-build 11
* libwget/dns.c (wget_dns_resolve): 'rc' was set but not used.
* libwget/http.c (wget_http_get_response_cb): 'rc' was set but not used.
* libwget/net.c (wget_tcp_connect): 'rc' was set but not used.
* libwget/ssl_gnutls.c (send_ocsp_request): 'rc' was set but not used.
2020-12-20 17:13:27 +01:00
d79a0da385 Fix type warnings on 32bit systems
* libwget/http.c (wget_http_get_response_cb): Fix printf format from %lX to %zX.
* libwget/ssl_openssl.c (wget_ssl_deinit): Fix sesslen to size_t.
* src/stats_site.c (print_csv_entry): Use %lld and cast to long long.
2020-12-06 16:10:32 +01:00
34319cee74 * libwget/http.c (wget_http_send_request): Fix unused variable without WITH_LIBNGHTTP2 2020-03-15 11:52:44 +01:00
d8cb7e2170 * libwget/http.c: Implement data upload for HTTP/2 2020-02-26 13:24:06 +01:00
b5f25ff67c * libwget/http.c (parse_proxies): Safer code with wget_strmemcpy 2020-02-25 11:07:23 +01:00
59d9ecd9c0 * Update copyright year to 2020 2020-01-10 00:33:02 +01:00
ab1200e48b * libwget/http.c (wget_http_get_response_cb): Fix implicit conversion -1 to size_t 2020-01-06 12:52:03 +01:00
5c42026ee4 Add --ignore-length handling
* include/wget/wget.h (wget_http_request): Add bool
response_ignorelength
(wget_http_response): Add bool length_inconsistent
* src/options.c: Add --ignore-length handling
* src/wget_options.h (config): Add bool ignore_length
(exit_status_e): Add WGET_EXIT_INCONSISTENT_LENGTH
* libwget/http.c (wget_http_response_cb): Set length_inconsistent
when mismatch with Content-Length
* src/wget.c (wget_send_request): Set ignore-length in request
* tests/test-ignore-length.c: test --ignore-length
* tests/Makefile.am: add test-ignore-length
* docs/wget2.md: Removed [unimplemented-option] from
--ignore-length
2019-11-05 21:43:17 +05:30
f77c718845 * libwget/http.c (wget_http_send_request): Fix hard-coded GET for HTTP/2 2019-10-28 14:15:02 +01:00
11a2703afe libwget/http.c (wget_http_get_response_cb): Don't ignore response body on header callback failure 2019-10-08 19:23:33 +02:00
998b74c18c Rename --retry-on-http-status to --retry-on-http-error
* docs/wget2.md: Likewise
* libwget/http.c: Likewise
* src/options.c: Likewise
* src/wget.c: Likewise
* src/wget_options.h: Likewise

Needed for backward compatibility with Wget 1.x.
2019-10-04 17:47:37 +02:00
91f7375b41 * libwget/http.c: Improve C99 compliancy 2019-09-18 17:16:01 +02:00
e066d46d3d * libwget/http.c: Use '!= 0' instead of !! 2019-09-18 12:55:28 +02:00
4b9edbad2d Update Copyright statements to be compatible with update-copyright module 2019-09-10 23:41:56 +02:00
0abce58d76 * libwget/http.c (wget_http_match_no_proxy): Check value from wget_vector_get() 2019-08-17 18:30:53 +02:00
955a1c289d * libwget/http.c (wget_http_create_request): Check memory allocation 2019-08-17 17:42:40 +02:00
2b4c141a1a * libwget/http.c (_parse_no_proxies): Check memory allocation 2019-08-16 17:42:02 +02:00
65ec4901b4 Remove global wget_iri_schemes[], add wget_iri_scheme enum
* include/wget/wget.h: Remove global wget_iri_schemes[],
  add wget_iri_scheme enum
* libwget/iri.c: Add struct iri_scheme,
  remove wget_iri_schemes and iri_ports,
  new function wget_iri_scheme_get_name(),
  fix code
* examples/check_url_types.c: Use comparison instead of wget_strcasecmp
* fuzz/libwget_iri_fuzzer.c: Use WGET_IRI_SCHEME_HTTPS instead of string
* libwget/http.c: Use wget_iri_scheme_get_name()
* libwget/http.h: Change scheme from string to wget_iri_scheme
* libwget/http_parse.c: Fix wget_http_get_scheme()
* src/blacklist.c: Fix hash_iri()
* src/host.c: Fix _host_hash()
* src/options.c: Use WGET_IRI_SCHEME_* instead of string
* src/stats_server.c: Use wget_iri_scheme for scheme member
* src/wget.c: Fix code
* src/wget_host.h: Use wget_iri_scheme fro scheme member
* unit-tests/test.c: Fix tests
2019-08-13 16:55:53 +02:00
6880c94e2e Move server stats code from library into wget2 tool
* fuzz/Makefile.am: Add src/stats_server.c
* src/Makefile.am: Likewise
* unit-tests/Makefile.am: Likewise
* include/wget/wget.h: Move `wget_server_stats_data` to src/stats_server.c,
  change wget_server_stats_callback,
  change wget_server_set_stats_callback(),
  add wget_tcp_get_ip(),
  remove host_ips_free()
* libwget/http.c: Move stats code to src/stats_server.c
* libwget/net.c: Add wget_tcp_get_ip()
* src/options.c: Move _hpkp_string() and stats_callback_server() to src/stats_server.c
* src/wget.c (main): Remove call to host_ips_free()
* src/wget_stats.h: Add prototypes of stats_server_add, server_stats_init, server_stats_exit
2019-08-12 16:14:48 +02:00
3835d892bb Rename HTTP parse callback types
* include/wget/wget.h: Rename HTTP parse callback types
* examples/getstream.c: Likewise
* libwget/http.c: Likewise
* libwget/http_highlevel.c: Likewise
2019-08-12 11:38:10 +02:00