f8f2f69a9f
Simplify vector API
...
* libwget/vector.c: Use wget_free as default destructor,
renamed _vec_insert_private() -> insert_element,
(insert_element): removed args 'size' and 'alloc',
removed the _noalloc() API,
removed the 'size' argument from functions,
added wget_vector_add_memdup() as convenience function,
renamed _compare() -> compare_element
* include/wget/wget.h: Removed _noalloc() functions
* fuzz/libwget_cookie_fuzzer.c: Amended to new API
* libwget/*.c: Likewise
* src/*.c: Likewise
* tests/*.c: Likewise
* unit-tests/*.c: Likewise
2019-06-26 16:43:20 +02:00
baa5ef562f
Add sc_prohibit_free to 'make syntax-check'
...
* cfg.mk: Add sc_prohibit_free rule plus exceptions
* examples/print_css_urls3.c: Remove unneeded comment
* fuzz/libwget_base64_fuzzer.c: Use wget_free() instead of free()
* fuzz/libwget_cookie_fuzzer.c: Likewise
* fuzz/libwget_css_url_fuzzer.c: Likewise
* fuzz/libwget_http_client_fuzzer.c: Likewise
* fuzz/libwget_utils_fuzzer.c: Likewise
* libwget/vector.c: Likewise
* src/options.c: Likewise
* src/wget_main.h: Likewise
* tests/test-compression.c: Likewise
* tests/test-cookies-http_state.c: Likewise
* tests/test-metalink.c: Likewise
* unit-tests/stringmap_perf.c: Likewise
This rule checks for calling free(). Instead, either use wget_free() or
the convenience macro xfree().
2019-06-24 16:04:23 +02:00
5e3c8e19be
* fuzz/wget_options_fuzzer.c: Fix compiler warning on MinGW
2019-06-24 12:34:20 +02:00
5a424bcea0
* fuzz/libwget_http_client_fuzzer.c: Add emulation of freeaddrinfo()
2019-06-23 15:38:25 +02:00
94d3969488
Remove --stats-all
...
* docs/wget2.md Remove docs for --stats-all
* src/options.c: Remove --stats-all and parse_stats_all()
* fuzz/wget_options_fuzzer.dict: remove --stats-all
* tests/Makefile.am: Remove test-stats-all
* tests/test-stats-all.c: Remove file
2019-06-21 15:52:40 +02:00
5230e1d165
Rewrite Site statistics --stats-site
...
* fuzz/Makefile.am: Remove ../src/stats_site.o
* include/wget/wget.h: Remove wget_stats_type_t
* po/POTFILES.in. Remove src/stats.c
* src/Makefile.am: Remove stats_site.c
* src/options.c (parse_stats_all): Remove site stats code,
(init): Add DNS stats code,
(deinit): Cleanup dns stats code
* src/stats.c: Remove file
* src/stats_site.c: iFix for new stats code
* src/wget.c (main): stats_print -> site_stats_print,
(add_statistics): config.stats_site -> config.stats_site_args,
(process_response): Likewise
* src/wget_options.h (struct config): Remove stats_site, add stats_site_args
* src/wget_stats.h: Remove stats_opt_t,
remove stats_print_func_t,
remove stats_callback_setter_t,
remove struct stats_opts,
remove stats_site_opts,
rename stats_init -> site_stats_init,
rename stats_exit -> site_stats_exit,
rename stats_print -> site_stats_print
* unit-tests/Makefile.am: Remove ../src/stats.o
2019-06-21 15:43:34 +02:00
619a7456c1
Rewrite Server statistics --stats-server
...
* fuzz/Makefile.am: Remove ../src/stats_server.o
* include/wget/wget.h:
remove wget_server_stats_t,
rename wget_stats_callback_t -> wget_server_stats_callback_t,
rename wget_tcp_set_stats_server -> wget_server_set_stats_callback,
add wget_server_stats_data_t,
add wget_server_stats_callback_t,
* libwget/http.c: Remove _stats_data_t,
add server_stats_callback,
add server_stats_ctx
* src/Makefile.am: Remove stats_server.c
* src/options.c (parse_stats_all): Remove server stats code,
add stats_callback_server(),
(init): Add DNS stats code,
(deinit): Cleanup dns stats code
* src/stats.c (stats_print_func_t): Remove print_server[],
remove WGET_STATS_TYPE_SERVER
* src/stats_server.c: Remove file
* src/wget_options.h (struct config): Remove stats_server, add stats_server_args
* src/wget_stats.h: Remove stats_server_opts
* unit-tests/Makefile.am: Remove ../src/stats_server.o
2019-06-21 15:33:06 +02:00
e5ef8c168f
Rewrite TLS statistics --stats-tls
...
* fuzz/Makefile.am: Remove ../src/stats_tls.o
* include/wget/wget.h: Add wget_tls_stats_data_t,
add wget_tls_stats_callback_t,
add wget_ssl_set_stats_callback_tls,
remove WGET_STATS_TYPE_TLS,
remove wget_tls_stats_t,
remove wget_tcp_set_stats_tls(),
remove wget_tcp_get_stats_tls()
* libwget/ssl_gnutls.c: Remove old stats code, add new stats code
* libwget/ssl_none.c: Add/remove function stubs
* libwget/ssl_wolfssl.c: Remove old stats code, add new stats code
* src/Makefile.am: Remove stats_tls.c
* src/options.c (parse_stats_all): Remove TLS stats code,
add stats_callback_tls(),
(init): Add TLS stats code,
* src/stats.c (stats_print_func_t): Remove print_tls[],
remove WGET_STATS_TYPE_TLS
* src/stats_tls.c: Remove file
* src/wget_options.h (struct config): Remove stats_tls, add stats_tls_args
* src/wget_stats.h: Remove stats_tls_opts
* unit-tests/Makefile.am: Remove ../src/stats_tls.o
2019-06-21 15:32:13 +02:00
6df1578283
Rewrite OCSP statistics --stats-ocsp
...
* fuzz/Makefile.am: Remove ../src/stats_ocsp.o
* include/wget/wget.h: Add wget_ocsp_stats_data_t,
add wget_ocsp_stats_callback_t,
add wget_ssl_set_stats_callback_ocsp,
remove WGET_STATS_TYPE_OCSP,
remove wget_ocsp_stats_t,
remove wget_tcp_set_stats_ocsp(),
remove wget_tcp_get_stats_ocsp()
* libwget/ssl_gnutls.c: Remove old stats code, add new stats code
* libwget/ssl_none.c: Add/remove function stubs
* libwget/ssl_wolfssl.c: Remove old stats code, add new stats code
* src/Makefile.am: Remove stats_ocsp.c
* src/options.c (parse_stats_all): Remove OCSP stats code,
add stats_callback_ocsp(),
(init): Add OCSP stats code,
* src/stats.c (stats_print_func_t): Remove print_ocsp[],
remove WGET_STATS_TYPE_OCSP
* src/stats_ocsp.c: Remove file
* src/wget_options.h (struct config): Remove stats_ocsp, add stats_ocsp_args
* src/wget_stats.h: Remove stats_ocsp_opts
* unit-tests/Makefile.am: Remove ../src/stats_ocsp.o
2019-06-21 15:32:13 +02:00
584c7f8801
Rewrite DNS statistics --stats-dns
...
* fuzz/Makefile.am: Remove ../src/stats_dns.o
* include/wget/wget.h: Add wget_dns_stats_data_t,
add wget_dns_stats_callback_t,
add wget_dns_set_stats_callback,
remove WGET_STATS_TYPE_DNS,
remove wget_dns_stats_t,
remove wget_dns_set_stats(),
remove wget_dns_get_stats()
* libwget/dns.c: Remove _stats_data_t,
amend struct wget_dns_st,
add wget_dns_set_stats_callback(),
remove wget_dns_get_stats()
* src/Makefile.am: Remove stats_dns.c
* src/options.c (parse_stats_all): Remove DNS stats code,
add stats_callback_dns(),
(init): Add DNS stats code,
* src/stats.c (stats_print_func_t): Remove print_dns[],
remove WGET_STATS_TYPE_DNS
* src/stats_dns.c: Remove file
* src/wget_options.h (struct config): Remove stats_dns, add stats_dns_args
* src/wget_stats.h: Remove stats_dns_opts
* tests/stats-test-util.h: Remove "human"
* unit-tests/Makefile.am: Remove ../src/stats_ocsp.o
2019-06-21 15:27:33 +02:00
73b7441e9a
Move wget_restrict_file_name() to src/utils.c
...
* fuzz/libwget_utils_fuzzer.c: Remove test code for wget_restrict_file_name()
* include/wget/wget.h: Remove wget_restrict_file_name() prototype
* libwget/utils.c: Remove wget_restrict_file_name()
* src/utils.c: Add wget_restrict_file_name()
* src/wget_utils.h: Add wget_restrict_file_name() prototype
* tests/Makefile.am: Add ../src/utils.c to libtest_la_SOURCES
* tests/libtest.c: Include ../src/wget_utils.h
2019-06-21 12:47:06 +02:00
ee13c9991c
Fix robots.txt code for stand-alone library API
...
* fuzz/libwget_robots_parse_fuzzer.c: Amend API call
* include/wget/wget.h: Amend wget_robots_parse() API
* libwget/robots.c: Amend wget_robots_parse() API,
check memory allocations
* libwget/test_linking_robots.c: New file
* src/wget.c (process_response): Amend call to wget_robots_parse()
* unit-tests/test.c (test_robots): Amend call to wget_robots_parse()
2019-06-19 16:13:14 +02:00
aa2b8f400f
Add sc_prohibit_sprintf and sc_prohibit_printf and fix code
...
* cfg.mk: New syntax rules sc_prohibit_sprintf and sc_prohibit_printf
* fuzz/*.c: Apply rules
* libwget/*.c: Likewise
* src/*.c: Likewise
* tests/*.[ch]: Likewise
* unit-tests/*.c: Likewise
2019-06-17 16:39:24 +02:00
52378f333e
* fuzz/main.c: Fix paths for WIN32
2019-05-18 22:32:03 +02:00
3bad69304d
* fuzz/wget_options_fuzzer.c: Slightly change magic number
...
Libfuzzer is too clever and detects the magic number used in the fuzzer.
This leads to weird behavior and false positives.
2019-04-25 22:47:18 +02:00
3da6b6cc06
* fuzz/*.in: Update fuzz corpora from OSS-fuzz
2019-04-25 21:49:46 +02:00
183752494c
* fuzz/get_ossfuzz_corpora: mkdir *.in directories
2019-04-25 21:48:54 +02:00
e68fd8b648
* fuzz/get_ossfuzz_corpora: Remove -f from unzip
2019-04-25 21:43:30 +02:00
41c307e817
* fuzz/libwget_http_client_fuzzer.c: Fix for OpenBSD
2019-02-20 11:39:05 +01:00
cdb3600791
Update copyrigght to 2019
2019-01-02 12:42:53 +01:00
2cd7ec9725
* fuzz/Makefile.am: Fix order of libraries for linking [skip ci]
2018-12-28 18:10:20 +01:00
1ec38598bb
Add valgrind suppressions for TravisCI (due to old libidn)
...
* fuzz/valgrind-suppressions: Add suppression
* tests/valgrind-suppressions: Add suppression
2018-10-21 19:41:37 +02:00
444808c260
Suppress some leaks detected by valgrind in fuzz/
...
* fuzz/main.c: Use --suppressions and --gen-suppressions
* fuzz/valgrind-suppressions: New suppression file
Some library's memory can't be freed. So we suppress what is needed
to let the CIs survive.
2018-10-19 16:40:09 +02:00
6510180733
* fuzz/*_fuzzer.in/*: Update fuzzer copora from OSS-Fuzz
2018-10-19 15:53:45 +02:00
1ff4b65436
* fuzz/get_ossfuzz_corpora: Use faster corpus download from OSS-Fuzz
2018-10-19 15:52:49 +02:00
2e2208be41
Fix issue with the number of bytes downloaded vs accounted for in the progress bar.
...
* include/wget/wget.h(wget_bar_slot_begin): Now accepts another
parameter, `new_file`.
* libwget/bar.c(wget_bar_slot_begin): Support `new_file` parameter.
Allows calling mehod multiple times for the same file.
* src/bar.c(bar_slot_begin): Same
* src/wget_bar.h: Same
* src/wget.c(_get_body): Pass only the difference (number of bytes
actually downloaded since last call) to the progress routines
2018-10-15 18:02:32 +02:00
74b3905b5b
Fix buffer overflow in -X/-I parsing
...
* src/options.c (set_char_prefix): Fix backslash handling
* fuzz/wget_options_fuzzer.in/: Add reproducer
This bug was introduced yesterday by commit
756e470de9
2018-09-01 13:49:55 +02:00
f6e762181d
Fix buffer overflow in -X/-I parsing
...
* src/options.c (set_char_prefix): Fix scratch buffer size
* fuzz/wget_options_fuzzer.in/: Add reproducer
This bug was introduced yesterday by commit
756e470de9
2018-09-01 13:31:54 +02:00
c4321b385e
* fuzz/libwget_base64_fuzzer.c: Silence scan-build
2018-08-21 13:15:00 +02:00
241b9a1d2d
Document wget_base64_get_(en|de)coded_length() and move to base64.c
...
* include/wget/wget.h: Declare both function as pure
* libwget/base64.c: Add both functions incl. documentation,
(wget_base64_decode_alloc): Remove hard-coded calculation,
(wget_base64_encode_alloc): Likewise.
* libwget/tls_session.c (_tls_session_save):
Use wget_base64_get_encoded_length() instead hard-coded calculation
* fuzz/libwget_base64_fuzzer.c:
Avoid both functions to be optimized out
2018-08-21 12:26:09 +02:00
71ef693415
* fuzz/fuzzer.h: Disable pedantic and unused warnings
2018-04-30 21:26:30 +02:00
3688ffb941
Update copyright to 2018
2018-04-30 20:52:11 +02:00
af88a7a2c3
Add G_GNUC_WGET_UNUSED to fuzz code
...
* fuzz/libwget_xml_parse_buffer_fuzzer.c: Likewise
* fuzz/main.c: Likewise
2018-04-30 20:08:06 +02:00
25a9a0151f
* fuzz/Makefile.am: Remove hard-coded compiler flags
2018-04-30 20:07:57 +02:00
c353a7d121
Exclude fuzz corpora from tarball
...
* fuzz/Makefile.am: Do not include corpora in tarball
* fuzz/main.c: SKIP if corpora directory isn't found (make check)
The fuzz corpora are thousands of files, not needed for a standard build
from a distribution tarball. The reproducers of former issues are being
included for regression testing.
2018-04-30 19:45:51 +02:00
129bc075b7
* fuzz/*.in: Update oss-fuzz corpora
2018-04-03 22:22:12 +02:00
8a31988b45
Remove param 'off' from wget_vector_create()
...
* libwget/vector.c (wget_vector_create): Remove param 'off',
(wget_vector_set_growth_policy): Add new function
* include/wget/wget.h: Add prototype for wget_vector_set_growth_policy()
* */*.c: Remove param 'off' from calls to wget_vector_create()
2018-03-25 19:05:45 +02:00
1cec729103
Fix oss-fuzz build
...
* fuzz/Makefile.am: Add ../src/stats_server.o ../src/stats_site.o to oss-fuzz rule
* fuzz/libwget_http_client_fuzzer.c: Fix pedantic C++ errors
2018-02-25 11:00:27 +01:00
83f37072ba
Move _shell_expand() into own source file
...
* src/Makefile.am: Add utils.c and wget_utils.h
* src/options.c: Remove _shell_expand()
* src/stats.c: Likewise
* src/utils.c: New file with shell_expand()
* src/wget_utils.h: Header file for shell_expand()
* fuzz/Makefile.am: Add utils.o
* unit-tests/Makefile.am: Likewise
2018-02-22 14:33:03 +01:00
8f53ba4fa6
Move dns stats code into stats_dns.c
2018-02-22 14:33:03 +01:00
309fd805a1
Add new fuzzer libwget_http_client_fuzzer
...
* fuzz/Makefile.am: Add libwget_http_client_fuzzer
* fuzz/libwget_http_client_fuzzer.c: New file
* fuzz/libwget_http_client_fuzzer.in: Initial corpora
* fuzz/run-clang.sh: Small cleanup
2018-02-02 19:46:22 +01:00
5a1af34ffb
* fuzz/wget_options_fuzzer.dict: Update with new options
2018-01-24 11:13:22 +01:00
fb48f74bd6
* fuzz/wget_options_fuzzer.in: Add new corpora
2018-01-24 11:13:19 +01:00
6c5d0ba63c
Fix --without-plugin-support build
...
* configure.ac: Check for dlsym and introduce $FUZZ_LIBS
* fuzz/Makefile.am: Use $FUZZ_LIBS instead of $LIBS
2018-01-03 17:13:43 +01:00
39dfeec1ae
Fixed problems pointed out in MR. Added tests and doxygen documentation.
...
* bootstrap.conf: Add canonicalize module.
* cfg.mk: Added exclusions to no new line at EOF for binary files used in tests.
* configure.ac: Check for libgpgme
* docs/wget2_manual.md: Added manual entries for --verify-sig, --gnupg-homedir, and new exit codes.
* fuzz/Makefile.am: Amend linker options
* include/wget/wget.h: Add error codes for GPG signature verification.
* po/POTFILES.in: Add new files.
* src/Makefile.am: Add src/gpgme.c and src/wget_gpgme.h
* src/gpgme.c: Using new error codes, and better signature verification failure detection.
* src/job.c (job_free): Free sig_filename
* src/options.c: Add new options --gnupg-homedir and --verify-sig
* src/wget.c: Verify the signature if possible, exit with the proper status code,
print error messages
* src/wget_gpgme.h: Add documentation.
* src/wget_job.h: Extend struct JOB
* src/wget_options.h: Add new status codes, add new members to struct config
* tests/Makefile.am: Added new tests.
* tests/gpg-test-util.h: Common GPG testing functionality (tries to verify a signature, expects exit code ... etc).
* tests/gpg/helloworld.txt: Text that has been signed in the .sig files.
* tests/gpg/helloworld.txt.{invalid,no-pub,missing,trusted}.sig: Signatures on helloworld.txt with properties described by their names.
* tests/gpg/openpgp-revocs.d/*: Revocation certs for the fake "homedir"
* tests/gpg/private-keys-v1.d/*: The not-so-private private keys used to sign the test file.
* tests/gpg/pubring.kbx: Fake gnupg homedir public key ring.
* tests/gpg/test-gpg-*.c: Tests.
* tests/valgrind-supressions: Supress failures due to oddness with GPGME.
* unit-tests/Makefile.am: Add src/gpgme.o to BASE_OBJS
2018-01-01 13:28:43 +01:00
ce5c9437a6
* fuzz/main.c: Fix path separator on MinGW
2017-12-31 00:10:51 +01:00
178b11b2da
* fuzz/*_fuzzer.in/*: Update fuzzer corpora
2017-12-08 22:11:46 +01:00
1742ca13e4
* fuzz/get_ossfuzz_corpora: Fix script to handle many corpora
2017-12-08 22:08:45 +01:00
932c3e7378
Update link libraries in Makefiles
...
* fuzz/Makefile.am: Update link libraries and remove redundant code
* libwget/Makefile.am: Likewise
* src/Makefile.am: Likewise
* tests/Makefile.am: Likewise
* unit-tests/Makefile.am: Likewise
2017-12-05 12:33:19 +01:00
5fd922ddf7
Fix 'make distcheck' for VPATH build
...
* fuzz/Makefile.am: Use $(srcdir) to find files
* tests/Makefile.am: Add stats-test-util.h to *_SOURCES where needed
2017-11-29 20:28:48 +01:00
