186 Commits

Author SHA1 Message Date
7da3819092 * fuzz/Makefile.am (oss-fuzz): Add -lgmp 2024-05-24 18:26:19 +02:00
4d1504aa53 * fuzz/Makefile.am: Add valgrind-suppressions to EXTRA_DIST 2024-05-21 13:55:50 +02:00
d55ecc7113 Update gnulib link libraries
* fuzz/Makefile.am: Update gnulib link libraries.
* libwget/Makefile.am: Likewise.
* src/Makefile.am: Likewise.
* tests/Makefile.am: Likewise.
* unit-tests/Makefile.am: Likewise.
2024-03-24 15:58:27 +01:00
188989e067 Update copyright year 2024-01-07 12:41:02 +01:00
774b17f7bc fuzz/libwget_cookie_fuzzer.c: Randomize keeping session cookies 2024-01-07 12:34:35 +01:00
a8d241db63 fuzz: Fix http_client fuzzer
Properly free the addrinfo structs, assuming there could be more than
one.
2023-09-07 19:55:49 +02:00
15ba1db248 Update copyrights 2023-08-31 12:57:30 +02:00
9f30b1a9c5 * fuzz/main.c: Replace VLA 2023-07-29 19:01:29 +02:00
c43d1d8f29 * fuzz/Makefile.am: Add -lz to fuzzer libs 2022-09-24 19:42:15 +02:00
abbc27c9f1 * fuzz/libwget_http_client_fuzzer.c: Silence clang warning 2022-06-26 14:01:12 +02:00
0f554ef216 * */Makefile.am: Reorder link libraries (libgnu.la first) 2022-03-05 16:23:15 +01:00
3dc7f71098 Update copyright year 2022-02-25 17:46:43 +01:00
f1650988e3 * fuzz/libwget_html_url_fuzzer.dict: Add 'data:' to dictionary 2021-05-30 19:36:04 +02:00
22162f82b2 Update copyright year 2021-01-22 21:58:38 +01:00
720d4bdbae * fuzz/*_fuzer.in/*: Update OSS-Fuzz corpora from upstream 2020-02-11 12:31:45 +01:00
83e32dfbb9 * fuzz/Makefile.am: Remove -all-static from AM_LDFLAGS 2020-02-11 12:24:18 +01:00
22eb51fc45 * fuzz/README.md: Update docs 2020-02-11 12:23:30 +01:00
59d9ecd9c0 * Update copyright year to 2020 2020-01-10 00:33:02 +01:00
0f624b16b5 Update Makefile.am for latest gnulib
* examples/Makefile.am: Amend LDADD assignment
* fuzz/Makefile.am: Likewise
* libwget/Makefile.am: Likewise
* src/Makefile.am: Likewise
* tests/Makefile.am: Likewise
* unit-tests/Makefile.am: Likwise
2020-01-07 15:46:12 +01:00
6d47c6b0db Fix --cut-file-get-vars creating dirs on single file download
* tests/test-cut-get-vars.c: Amend to reproduce issue #490.
* src/blacklist.c (get_local_filename_real): Generate basename
  instead of filename, without query.
* include/wget/wget.h: Add flag WGET_IRI_WITH_QUERY.
  Add param 'flags' to wget_iri_get_basename.
* libwget/iri.c: Add param 'flags' to wget_iri_get_basename.
  Implement flag WGET_IRI_WITH_QUERY.
* fuzz/libwget_iri_fuzzer.c (test): Use WGET_IRI_WITH_QUERY.

Closes #490
2019-12-07 17:29:48 +01:00
f3e1d61a25 Rename wget_iri_get_filename -> wget_iri_get_basename
* fuzz/libwget_iri_fuzzer.c: Likewise.
* include/wget/wget.h: Likewise.
* libwget/iri.c: Likewise.
* src/blacklist.c: Likewise.
2019-12-07 15:40:53 +01:00
2ec7c7e290 Deduplicate code for creating robots.txt jobs
* src/wget.c (queue_url_from_local): Remove duplicated code,
  (queue_url_from_remote): Likewise,
  (set_file_metadata): Amend calling wget_iri_get_connection_part()
* include/wget/wget.h (wget_iri_parse_base): Add 'const' to 'base' param,
  (wget_iri_get_connection_part): Add 'const' to 'iri' param,
  add second param 'buf'
* fuzz/libwget_iri_fuzzer.c (test): Amend calling wget_iri_get_connection_part()
* libwget/iri.c (wget_iri_get_connection_part): Add param 'buf',
  add 'const' to 'iri' param, amend docs,
  (wget_iri_relative_to_abs): Add 'const' to 'base' param,
  amend calling wget_iri_get_connection_part(),
  (wget_iri_parse_base): Add 'const' to 'iri' param
* src/host.c (host_add_robotstxt_job): Add code removed in wget.c
* src/wget_host.h: Don't include wget_blacklist.h,
  amend signature of host_add_robotstxt_job()
* src/wget_job.h (struct JOB): Add 'const' to 'blacklist_entry'
2019-10-26 18:48:33 +02:00
4b9edbad2d Update Copyright statements to be compatible with update-copyright module 2019-09-10 23:41:56 +02:00
65ec4901b4 Remove global wget_iri_schemes[], add wget_iri_scheme enum
* include/wget/wget.h: Remove global wget_iri_schemes[],
  add wget_iri_scheme enum
* libwget/iri.c: Add struct iri_scheme,
  remove wget_iri_schemes and iri_ports,
  new function wget_iri_scheme_get_name(),
  fix code
* examples/check_url_types.c: Use comparison instead of wget_strcasecmp
* fuzz/libwget_iri_fuzzer.c: Use WGET_IRI_SCHEME_HTTPS instead of string
* libwget/http.c: Use wget_iri_scheme_get_name()
* libwget/http.h: Change scheme from string to wget_iri_scheme
* libwget/http_parse.c: Fix wget_http_get_scheme()
* src/blacklist.c: Fix hash_iri()
* src/host.c: Fix _host_hash()
* src/options.c: Use WGET_IRI_SCHEME_* instead of string
* src/stats_server.c: Use wget_iri_scheme for scheme member
* src/wget.c: Fix code
* src/wget_host.h: Use wget_iri_scheme fro scheme member
* unit-tests/test.c: Fix tests
2019-08-13 16:55:53 +02:00
8757741b6f * .gitlab-ci.yml: Add FreeBSD runner 2019-08-13 10:21:27 +02:00
6880c94e2e Move server stats code from library into wget2 tool
* fuzz/Makefile.am: Add src/stats_server.c
* src/Makefile.am: Likewise
* unit-tests/Makefile.am: Likewise
* include/wget/wget.h: Move `wget_server_stats_data` to src/stats_server.c,
  change wget_server_stats_callback,
  change wget_server_set_stats_callback(),
  add wget_tcp_get_ip(),
  remove host_ips_free()
* libwget/http.c: Move stats code to src/stats_server.c
* libwget/net.c: Add wget_tcp_get_ip()
* src/options.c: Move _hpkp_string() and stats_callback_server() to src/stats_server.c
* src/wget.c (main): Remove call to host_ips_free()
* src/wget_stats.h: Add prototypes of stats_server_add, server_stats_init, server_stats_exit
2019-08-12 16:14:48 +02:00
12f1a03693 Changed G_GNUC_WGET prefix to WGET_GCC
* include/wget/wget.h: Changed G_GNUC_WGET prefix to WGET_GCC
* */*.[ch]: Likewise
2019-08-08 17:13:24 +02:00
6485e41497 Rename wget_hsts_db_t -> wget_hsts_db
* include/wget/wget.h: Rename wget_hsts_db_t -> wget_hsts_db,
* fuzz/libwget_hsts_fuzzer.c: Likewise
* libwget/hsts.c: Likewise
* src/wget_options.h: Likewise
* tests/test-plugin-dummy.c: Likewise
* unit-tests/test.c: Likewise
2019-08-07 15:33:37 +02:00
ab98597170 Rename wget_hpkp_db_t -> wget_hpkp_db and wget_hpkp_t -> wget_hpkp
* include/wget/wget.h: Rename wget_hpkp_db_t -> wget_hpkp_db,
  rename wget_hpkp_t -> wget_hpkp
* fuzz/libwget_hpkp_fuzzer.c: Likewise
* libwget/hpkp.c: Likewise
* libwget/http_parse.c: Likewise
* libwget/ssl_gnutls.c: Likewise
* libwget/ssl_wolfssl.c: Likewise
* src/wget_options.h: Likewise
* tests/test-plugin-dummy.c: Likewise
* unit-tests/test.c: Likewise
2019-08-07 15:33:27 +02:00
8843b8f23e Added support for ax_code_coverage version 32
* Makefile.am: Add code coverage rules as documented in ax_code_coverage.m4
* cfg.mk: Exclude m4 files from sc_GPL_version
* configure.ac: Add AX_CODE_COVERAGE
* fuzz/Makefile.am: Add coverage flags
* libwget/Makefile.am: Likewise
* src/Makefile.am: Likewise
* tests/Makefile.am: Likewise
* m4/ax_ac_append_to_file.m4: New file
* m4/ax_ac_print_to_file.m4: New file
* m4/ax_add_am_macro_static.m4: New file
* m4/ax_am_macros_static.m4: New file
* m4/ax_check_gnu_make.m4: New file
* m4/ax_code_coverage.m4: New file
* m4/ax_file_escapes.m4: New file
2019-08-06 15:38:47 +02:00
e58c2873d7 Rename wget_bar_t -> wget_bar
* include/wget/wget.h: Rename wget_bar_t -> wget_bar
* fuzz/libwget_bar_fuzzer.c: Likewise
* libwget/bar.c: Likewise
* src/bar.c: Likewise
* src/wget_bar.h: Likewise
* unit-tests/test.c: Likewise
2019-07-19 17:05:47 +02:00
acf1ec76bc Rename wget_robots_t -> wget_robots
* include/wget/wget.h: Rename wget_robots_t -> wget_robots
* fuzz/libwget_robots_parse_fuzzer.c: Likewise
* libwget/robots.c: Likewise
* libwget/test_linking_robots.c: Likewise
* src/wget_host.h: Likewise
* unit-tests/test.c: Likewise
2019-07-19 17:05:47 +02:00
41e45232d8 Rename wget_metalink_*_t -> wget_metalink_*
* include/wget/wget.h: Rename wget_metalink_*_t -> wget_metalink_*
* libwget/metalink.c: Likewise
* src/job.c: Likewise
* src/wget.c: Likewise
* src/wget_job.h: Likewise
2019-07-19 17:05:47 +02:00
e6aba6bb09 Rename wget_http_connection_t -> wget_http_connection
* include/wget/wget.h: Rename wget_http_connection_t -> wget_http_connection
* examples/check_url_types.c: Likewise
* examples/http_get.c: Likewise
* examples/http_get2.c: Likewise
* examples/http_multi_get.c: Likewise
* examples/websequencediagram.c: Likewise
* examples/websequencediagram_high.c: Likewise
* fuzz/libwget_http_client_fuzzer.c: Likewise
* libwget/http.c: Likewise
* libwget/http_highlevel.c: Likewise
* libwget/http_parse.c: Likewise
* libwget/ssl_gnutls.c: Likewise
* src/wget.c: Likewise
* src/wget_job.h: Likewise
2019-07-19 16:47:01 +02:00
35c18afa13 Rename wget_http_request_t -> wget_http_request
* include/wget/wget.h: Rename wget_http_request_t -> wget_http_request
* examples/http_get2.c: Likewise
* examples/http_multi_get.c: Likewise
* examples/websequencediagram.c: Likewise
* fuzz/libwget_http_client_fuzzer.c: Likewise
* libwget/http.c: Likewise
* libwget/http_highlevel.c: Likewise
* libwget/http_parse.c: Likewise
* libwget/ssl_gnutls.c: Likewise
* src/wget.c: Likewise
* unit-tests/test.c: Likewise
2019-07-19 16:44:21 +02:00
da75b622d6 Rename wget_http_response_t -> wget_http_response
* include/wget/wget.h: Rename wget_http_response_t -> wget_http_response
* examples/batch_loader.c: Likewise
* examples/check_url_types.c: Likewise
* examples/getstream.c: Likewise
* examples/http_get.c: Likewise
* examples/http_get2.c: Likewise
* examples/http_multi_get.c: Likewise
* examples/websequencediagram.c: Likewise
* examples/websequencediagram_high.c: Likewise
* fuzz/libwget_http_client_fuzzer.c: Likewise
* fuzz/libwget_http_parse_fuzzer.c: Likewise
* libwget/http.c: Likewise
* libwget/http_highlevel.c: Likewise
* libwget/http_parse.c: Likewise
* libwget/ssl_gnutls.c: Likewise
* src/gpgme.c: Likewise
* src/stats_site.c: Likewise
* src/wget.c: Likewise
* src/wget_gpgme.h: Likewise
* src/wget_stats.h: Likewise
* tests/test-cookies-http_state.c: Likewise
* unit-tests/test-cookies-http_state.c: Likewise
* unit-tests/test.c: Likewise
2019-07-19 16:41:21 +02:00
8c47901ef8 Rename wget_html_parsed_result_t -> wget_html_parsed_result
* include/wget/wget.h: Rename wget_html_parsed_result_t -> wget_html_parsed_result
* examples/check_url_types.c: Likewise
* examples/print_html_urls.c: Likewise
* fuzz/libwget_html_url_fuzzer.c: Likewise
* libwget/html_url.c: Likewise
* src/wget.c: Likewise
2019-07-19 16:14:33 +02:00
ae5c2cb664 Rename wget_netrc_db_t -> wget_netrc_db
* include/wget/wget.h: Rename wget_netrc_db_t -> wget_netrc_db
* fuzz/libwget_netrc_fuzzer.c: Likewise
* libwget/netrc.c: Likewise
* src/wget_options.h: Likewise
* unit-tests/test.c: Likewise
2019-07-19 16:02:49 +02:00
e61d3298e5 Rename wget_ocsp_db_t -> wget_ocsp_db
* include/wget/wget.h: Rename wget_ocsp_db_t -> wget_ocsp_db
* examples/http_multi_get.c: Likewise
* fuzz/libwget_ocsp_fuzzer.c: Likewise
* libwget/ocsp.c: Likewise
* libwget/plugin.c: Likewise
* libwget/ssl_gnutls.c: Likewise
* libwget/ssl_wolfssl.c: Likewise
* src/plugin.c: Likewise
* src/wget_options.h: Likewise
* src/wget_plugin.h: Likewise
* tests/test-plugin-dummy.c: Likewise
2019-07-19 15:59:58 +02:00
30269fd5c5 Rename wget_tls_session_db_t -> wget_tls_session_db
* include/wget/wget.h: Rename wget_tls_session_db_t -> wget_tls_session_db
* fuzz/libwget_tlssess_fuzzer.c: Likewise
* libwget/ssl_gnutls.c: Likewise
* libwget/ssl_wolfssl.c: Likewise
* libwget/tls_session.c: Likewise
* src/wget_options.h: Likewise
2019-07-18 17:05:24 +02:00
7151a01a98 Rename wget_cookie_t -> wget_cookie
* include/wget/wget.h: Rename wget_cookie_t -> wget_cookie
* fuzz/libwget_cookie_fuzzer.c: Likewise
* libwget/cookie.c: Likewise
* libwget/http_parse.c: Likewise
* unit-tests/test.c: Likewise
2019-07-18 13:14:26 +02:00
618f49160d Rename wget_cookie_db_t -> wget_cookie_db
* include/wget/wget.h: Rename wget_cookie_db_t -> wget_cookie_db
* examples/http_get2.c: Likewise
* fuzz/libwget_cookie_fuzzer.c: Likewise
* libwget/cookie.c: Likewise
* libwget/http_highlevel.c: Likewise
* libwget/init.c: Likewise
* src/wget_options.h: Likewise
* tests/test-cookies-http_state.c: Likewise
* unit-tests/test-cookies-http_state.c: Likewise
* unit-tests/test.c: Likewise
2019-07-18 13:11:55 +02:00
50be5af992 Rename wget_iri_t -> wget_iri
* include/wget/wget.h: Rename wget_iri_t -> wget_iri
* examples/*.c: Likewise
* fuzz/*.c: Likewise
* libwget/*.c: Likewise
* src/*.c: Likewise
* tests/*.c: Likewise
* unit-tests/*.c: Likewise
2019-07-18 13:10:27 +02:00
bdbcbb7992 Rename wget_vector_t -> wget_vector
* include/wget/wget.h: Rename wget_vector_t -> wget_vector
* examples/print_css_urls3.c: Likewise
* fuzz/*.c: Likewise
* libwget/*.c: Likewise
* src/*.c: Likewise
* tests/*.c: Likewise
* unit-tests/*.c: Likewise
2019-07-18 12:35:37 +02:00
db50c6a801 Rename wget_buffer_t -> wget_buffer
* include/wget/wget.h: Rename wget_buffer_t -> wget_buffer
* examples/*.c: Likewise
* fuzz/libwget_iri_fuzzer.c: Likewise
* libwget/*.c: Likewise
* src/*.c: Likewise
* tests/*.c: Likewise
* unit-tests/*.c: Likewise
2019-07-18 12:30:57 +02:00
8363aad83d * fuzz/*.in: Update fuzz corpora from OSS-Fuzz 2019-07-12 12:54:40 +02:00
d59dd30467 * fuzz/README.md: clang-6.0 -> clang 2019-07-12 12:50:27 +02:00
f8f2f69a9f Simplify vector API
* libwget/vector.c: Use wget_free as default destructor,
  renamed _vec_insert_private() -> insert_element,
  (insert_element): removed args 'size' and 'alloc',
  removed the _noalloc() API,
  removed the 'size' argument from functions,
  added wget_vector_add_memdup() as convenience function,
  renamed _compare() -> compare_element
* include/wget/wget.h: Removed _noalloc() functions
* fuzz/libwget_cookie_fuzzer.c: Amended to new API
* libwget/*.c: Likewise
* src/*.c: Likewise
* tests/*.c: Likewise
* unit-tests/*.c: Likewise
2019-06-26 16:43:20 +02:00
baa5ef562f Add sc_prohibit_free to 'make syntax-check'
* cfg.mk: Add sc_prohibit_free rule plus exceptions
* examples/print_css_urls3.c: Remove unneeded comment
* fuzz/libwget_base64_fuzzer.c: Use wget_free() instead of free()
* fuzz/libwget_cookie_fuzzer.c: Likewise
* fuzz/libwget_css_url_fuzzer.c: Likewise
* fuzz/libwget_http_client_fuzzer.c: Likewise
* fuzz/libwget_utils_fuzzer.c: Likewise
* libwget/vector.c: Likewise
* src/options.c: Likewise
* src/wget_main.h: Likewise
* tests/test-compression.c: Likewise
* tests/test-cookies-http_state.c: Likewise
* tests/test-metalink.c: Likewise
* unit-tests/stringmap_perf.c: Likewise

This rule checks for calling free(). Instead, either use wget_free() or
the convenience macro xfree().
2019-06-24 16:04:23 +02:00
5e3c8e19be * fuzz/wget_options_fuzzer.c: Fix compiler warning on MinGW 2019-06-24 12:34:20 +02:00