Commit Graph

143 Commits

Author SHA1 Message Date
dc8966d906 Don't log URI userinfo to logs
* include/wget/wget.h (struct wget_iri_st): Add safe_uri member.
* libwget/iri.c: New function create_safe_uri(),
  (wget_iri_unescape_url_inline): Check iri->userinfo,
  (wget_iri_parse): Likewise,
  (wget_iri_clone): Likewise,
  (wget_iri_relative_to_abs): Remove debug logs,
  (wget_iri_set_scheme): Check iri->userinfo.
* src/blacklist.c: Log safe_uri instead of uri.
* src/host.c: Likewise.
* src/options.c: Likewise.
* src/stats_site.c: Likewise.
* src/wget.c: Likewise,
  (queue_url_from_remote): Move redirection check further down.
* unit-tests/test.c (test_iri_parse): Add checks for safe_uri,
  (test_iri_parse): Add safe_uri to IRI comparison,
  Print safe_uri and password.

Co-authored-by: Tim Rühsen <tim.ruehsen@gmx.de>
2024-10-03 17:25:38 +02:00
9aeab55d09 Fix --no-parent for denormalized paths
* libwget/iri.c (wget_iri_parse): Normalize path part of URL.
* unit-tests/test.c (test_iri_parse): Add test with denormalized path.
2024-03-28 18:17:26 +01:00
d55ecc7113 Update gnulib link libraries
* fuzz/Makefile.am: Update gnulib link libraries.
* libwget/Makefile.am: Likewise.
* src/Makefile.am: Likewise.
* tests/Makefile.am: Likewise.
* unit-tests/Makefile.am: Likewise.
2024-03-24 15:58:27 +01:00
90549f72db * unit-tests/test.c (test_robots): Fix memory leak 2024-03-17 19:10:39 +01:00
a9300f643d * unit-tests/test.c (test_robots): Add new tests 2024-03-15 18:13:24 +01:00
188989e067 Update copyright year 2024-01-07 12:41:02 +01:00
1bab5454f9 Fix IPv6 address representation
* libwget/dns.c (resolve): Fix IPv6 in debug message,
  (wget_dns_cache_ip): Fix IPv6 in error message,
  (wget_dns_resolve): Fix IPv6 in debug message.
* libwget/dns_cache.c (wget_dns_cache_get): Fix IPv6 in debug message.
* libwget/hsts.c (hsts_db_add_entry): Fix IPv6 in debug message.
* libwget/http.c (wget_http_create_request): Fix IPv6 in Host header,
  (establish_proxy_connect): Fix IPv6 in CONNECT and in Host header.
* libwget/ip.c (wget_ip_is_family): Cosmetic change.
* libwget/iri.c (wget_iri_get_connection_part): Fix IPv6 in URL creation.
* libwget/net.c (debug_addr): Fix IPv6 in debug message.
* src/options.c (stats_callback_dns): Fix IPv6 in output.
* src/wget.c (test_modify_hsts): Fix IPv6 in console output,
  (http_create_request): Fix IPv6 in Referer header.
* unit-tests/test.c (test_iri_parse): Add two IPv6 parsing tests.
* libwget/Makefile.am (libwget_dnscache.la): Link with libwget_ip.la,
  (libwget_hsts.la): Link with libwget_ip.la.
2023-12-17 18:52:27 +01:00
7014ef7a51 * unit-tests/test.c: Add tests for wget_http_parse_full_date() 2023-11-17 19:40:51 +01:00
15ba1db248 Update copyrights 2023-08-31 12:57:30 +02:00
7b31ac3a50 * unit-tests/test-parse-html.c: Replace VLA 2023-07-29 19:01:52 +02:00
bee185be5b * unit-tests/test-dl.c: Replace VLA 2023-07-29 19:01:43 +02:00
c876010332 * unit-tests/test-decompress.c: Replace VLA 2023-07-29 19:01:38 +02:00
25813e3b6a * unit-tests/test.c: Replace VLA 2023-07-29 19:01:35 +02:00
68a11b7287 Add CIDR support in 'no_proxy' for IPv6
* include/wget/wget.h: Fix http_get_no_proxy -> wget_http_get_no_proxy.
* libwget/http.c: New function cidr_v6_match(),
  check for ipv6 CIDRs in wget_http_match_no_proxy().
* unit-tests/test.c (test_match_no_proxy): Add IPv6 CIDR tests.
2023-06-17 23:50:53 +02:00
6ab47257f0 CIDR support added for no_proxy
* libwget/http.c: Add function wget_http_cidr_match to support
  CIDR IP method for no_proxy
* unit-tests/test.c: Add testcases to verify the functionality
  of the above function

Fixes #615
2023-05-22 17:46:02 +00:00
2f55a2a1ca * unit-tests/test.c (test_robots): Fix test error message [skip ci] 2022-07-23 15:53:33 +02:00
b89f8f8079 Handle comments in robots.txt correctly
* libwget/robots.c (wget_robots_parse): Check for # as terminator.
* unit-tests/test.c (test_robots): Fix tests, add new test for comments.
2022-07-02 12:53:01 +02:00
791affdaaa Fix wget_buffer_trim() for strings with only whitespaces
* libwget/buffer.c: Don't access `start - 1`
* unit-tests/test.c: Add tests

Co-authored-by: Tim Rühsen <tim.ruehsen@gmx.de>
2022-06-26 13:45:17 +02:00
24c7691fa0 * unit-tests/test.c (check): Exit if any of the basic tests fail
Found-by: scan-build
2022-06-25 19:25:11 +02:00
c4e9c0966d * unit-tests/test.c (test_buffer): Add new test 2022-06-25 19:21:36 +02:00
0f554ef216 * */Makefile.am: Reorder link libraries (libgnu.la first) 2022-03-05 16:23:15 +01:00
3dc7f71098 Update copyright year 2022-02-25 17:46:43 +01:00
f9491cae37 * unit-tests/test.c (test_cookies): Fix false reporting of a PSL error 2022-02-19 13:44:17 +01:00
97afd2138d Allow spaces and \ escaping in passwords in .netrc
This is Wget1.x compatible.

* docs/wget2.md: Document the new behavior.
* libwget/netrc.c (wget_netrc_db_load): Amended the code to allow \ and "".
* unit-tests/test.c (test_netrc): Add tests.
2022-02-13 19:27:42 +01:00
7e11dfc3b0 * unit-tests/test.c (test_cookies): Check wget_cookie_db_load_psl() return value 2022-01-18 11:15:44 +01:00
73df033b52 * unit-tests/test.c: Free memory in test_parse_header_line() 2021-05-16 14:42:26 +02:00
ccc0092a88 Add test case for Content-Disposition
* libwget/http_parse.c: Update comment with BNF of RFC 6266.
* unit-tests/test.c: Add test for a filename containing space.
2021-03-27 19:59:03 +01:00
22162f82b2 Update copyright year 2021-01-22 21:58:38 +01:00
b2c6a4728e Fix warning when converting void * to function pointer
* unit-tests/test-dl.c (test_fn_check): Add #pragma to suppress
  -pedantic when converting void * to function pointer.

POSIX requires a conversion from 'void *' into a function pointer to work.
But -pedantic throws 'ISO C forbids conversion of object pointer to function
pointer type'.
2020-04-12 19:34:35 +02:00
439f463f12 Fix strict aliasing violation
* unit-tests/test-dl.c (test_fn_check): Avoid strict aliasing
violation.
2020-04-12 19:34:29 +02:00
beb557ea84 Fix undefined behaviour in test_buffer_printf
* unit-tests/test.c (test_buffer_printf): Cast argument to printf
function to correct type.
2020-04-12 16:12:48 +02:00
473a4da2bc Let wget_strmemcpy return the number of copied bytes
* include/wget/wget.h: Set return type of wget_strmemcpy to size_t.
* libwget/mem.c (wget_strmemcpy): Return length of destination string.
* unit-tests/test.c: Amend tests of wget_strmemcpy.
2020-02-24 16:30:02 +01:00
59d9ecd9c0 * Update copyright year to 2020 2020-01-10 00:33:02 +01:00
0f624b16b5 Update Makefile.am for latest gnulib
* examples/Makefile.am: Amend LDADD assignment
* fuzz/Makefile.am: Likewise
* libwget/Makefile.am: Likewise
* src/Makefile.am: Likewise
* tests/Makefile.am: Likewise
* unit-tests/Makefile.am: Likwise
2020-01-07 15:46:12 +01:00
99c3138e1c * unit-tests/test.c: Fix implicit conversions -1 to size_t 2020-01-06 13:03:12 +01:00
d3ee4089dd * unit-tests/test.c (test_mem): Fix buffer overflow in test 2019-12-19 17:10:36 +01:00
149a811543 Fix type of 2nd param of wget_stringmap_iterator_next() to void **
* include/wget/wget.h: Likewise
* unit-tests/test.c: Likewise
2019-10-26 18:48:33 +02:00
77c3c33ec8 Use int64_t for time_t to fix 32bit time_t limitations
* include/wget/wget.h: Use int64_t instead of time_t
* libwget/hpkp.c: Likewise
* libwget/hsts.c: Likewise
* libwget/http_parse.c: Likewise
* libwget/ocsp.c: Likewise
* libwget/ssl_gnutls.c: Likewise
* libwget/tls_session.c: Likewise
* src/options.c: Likewise
* src/stats_site.c: Likewise
* src/wget.c: Likewise
* tests/libtest.c: Likewise
* tests/libtest.h: Likewise
* tests/test-plugin-dummy.c: Likewise
* unit-tests/test.c: Likewise
2019-09-23 21:17:38 +02:00
baf385f48d * unit-tests/test-dl.c: Add missing len param to plugin function call 2019-09-23 15:11:54 +02:00
5645e709a2 Add unit test test-decompress.c
* unit-tests/Makefile.am: Add test-decompress.c
* unit-tests/test-decompress.c: New file
2019-09-21 22:16:25 +02:00
08d445e3f6 Use bool instead of char in HSTS code
* include/wget/wget.h: Change param to bool in wget_hsts_db_add_fn(),
  (struct wget_http_response_st): Use bool for hsts_include_subdomains and keep_alive,
  change param to bool in wget_http_parse_strict_transport_security(),
  change param to bool in wget_http_parse_connection()
* libwget/hsts.c: change param to bool in new_hsts(),
  change param to bool in wget_hsts_db_add()
* libwget/http_parse.c: change param to bool in wget_http_parse_strict_transport_security(),
  change param to bool in wget_http_parse_connection()
* tests/test-plugin-dummy.c: change param to bool in test_hsts_db_add()
* unit-tests/test.c (test_hsts): Change type of include_subdomains to bool
2019-09-18 12:44:26 +02:00
4b9edbad2d Update Copyright statements to be compatible with update-copyright module 2019-09-10 23:41:56 +02:00
a4dac96cf3 Update cfg.mk. Add comments for future
* cfg.mk: Multiple changes.
  (sc_GPL_version): Don't exclude all files when only one has a parsing
  issue
  (sc_po_check): Tests should always output English. Their output is for
  developers, not users
  (sc_prohibit_magic_number_exit): tests/test-plugin.c has a parsing
  issue that has been reported to gnulib. Others should be checked.
  (sc_trailing_blank): Fixed the relevant files instead of ignoring them
  (sc_two_space_separator_in_usage): Doesn't seem to trigger on removal
  (sc_prohibit_empty_lines_at_EOF): Same
  (sc_prohibit_sprintf): Same
  (sc_prohibit_printf): Re-arrange within file
  (sc_prohibit_free): Same
  (sc_prohibit_alloc): Same
  (sc_gettext_printf): Same
* docs/DoxygenLayout.xml: Remove trailing whitespaces
* docs/libwget.doxy.in: Same
* tests/libtest.c: Don't internationalize test suite strings
* tests/test-*.c: Use EXIT_FAILURE and EXIT_SUCCESS instead of 1 and 0
2019-09-10 23:02:05 +02:00
4c3945641a Fix hashing with WolfTLS backend
* include/wget/wget.h: Change param for wget_hash_init(),
  change param and return type for wget_hash_deinit()
* libwget/hashfile.c: Rewrite hashing functions for WITH_LIBWOLFCRYPT
* unit-tests/test.c: Add tests for incremental hashing
2019-08-29 22:36:35 +02:00
54c32f5627 Cleanup + amend return vales of buffer functions
* include/wget/wget.h: Add wget_buffer.error
* libwget/buffer.c: Amend docs,
  change return type of wget_buffer_init(),
  amend wget_buffer_alloc(),
  set buf.error in _buffer_realloc()
* libwget/printf.c: Check buf.error, return (size_t)-1
* unit-tests/test.c: Amend tests appropriately
2019-08-21 21:46:20 +02:00
0bf3c3cdfd * unit-tests/Makefile.am: Add -DMALLOC_RETURNS_NONNULL to AM_CFLAGS and AM_CPPFLAGS 2019-08-16 15:24:19 +02:00
370997e337 * unit-tests/test-dl.c (string_vector_check): Fix va_list leak 2019-08-16 15:24:19 +02:00
c3075a1e21 Introduce nullability support for clang's static analyzer
* configure.ac: Add -Wno-nullability-extension -Wno-nullability-completeness to WARN_CFLAGS
* include/wget/wget.h: Add CLANG_VERSION_AT_LEAST,
  fix attribute defines for clang,
  make use of attributes by using static inline functions for wget allocation,
  define NULLABLE for clang
* libwget/xalloc.c: Rename wget allocation function pointers
* src/options.c: Likewise
* unit-tests/test.c: Likewise
2019-08-16 15:24:19 +02:00
aa60e2333f Change return type of wget_hpkp_get_n_pins() to int
* include/wget/wget.h: Change return type of wget_hpkp_get_n_pins() to int
* libwget/hpkp.c: Likewise
* unit-tests/test.c: Adjust code, cleanup
2019-08-13 16:55:59 +02:00
65ec4901b4 Remove global wget_iri_schemes[], add wget_iri_scheme enum
* include/wget/wget.h: Remove global wget_iri_schemes[],
  add wget_iri_scheme enum
* libwget/iri.c: Add struct iri_scheme,
  remove wget_iri_schemes and iri_ports,
  new function wget_iri_scheme_get_name(),
  fix code
* examples/check_url_types.c: Use comparison instead of wget_strcasecmp
* fuzz/libwget_iri_fuzzer.c: Use WGET_IRI_SCHEME_HTTPS instead of string
* libwget/http.c: Use wget_iri_scheme_get_name()
* libwget/http.h: Change scheme from string to wget_iri_scheme
* libwget/http_parse.c: Fix wget_http_get_scheme()
* src/blacklist.c: Fix hash_iri()
* src/host.c: Fix _host_hash()
* src/options.c: Use WGET_IRI_SCHEME_* instead of string
* src/stats_server.c: Use wget_iri_scheme for scheme member
* src/wget.c: Fix code
* src/wget_host.h: Use wget_iri_scheme fro scheme member
* unit-tests/test.c: Fix tests
2019-08-13 16:55:53 +02:00