After the previous commit it now uses -Wl,--no-whole-archive
instead of patching libtool in the build system. This is more
portable for other libtool implementations other than the one
by GNU.
* Makefile.am: Remove data/ from SUBDIRS
* configure.ac: Remove data/Makefile from AC_CONFIG_FILES
* tests/Makefile.am: Remove WGET_DATADIR from DEFS
* unit-tests/Makefile.am: Likewise
* data/effective_tld_names.dat: Move to unit-tests/files/public_suffix_list.dat
and update from upstream
* unit-tests/test.c (test_cookies): Use public_suffix_list.dat
* Makefile.am: Add README.md to EXTRA_DIST
* configure.ac: Generate conditionals WITH_DOXYGEN and WITH_PANDOC
* docs/Makefile.am: Use conditionals for building
* Makefile.am: Fix make target 'fuzz-coverage'
* configure.ac: Enable building static library by default,
add option --enable-fuzzing
* fuzz/Makefile.am: Add wget_options_fuzzer,
check for FUZZING (set with --enable-fuzzing)
* fuzz/README.md: Amend the text
* fuzz/main.c: Use printf() instead of wget_info_printf()
* fuzz/run-clang.sh: Use fuzzer binaries built by 'make'
* fuzz/wget_options_fuzzer.c: New fuzzer
* fuzz/wget_options_fuzzer.dict: New fuzzer dictionary
* fuzz/wget_options_fuzzer.in/*: Initial fuzz corpora
* libwget/net.c: Skip IP address resolution when fuzzing
* src/log.c: Don't create files when fuzzing,
don't print to console when fuzzing
* src/options.c: Add set_exit_status() and get_exit_status(),
don't print --help / --version to console when fuzzing,
do not call exit() - return error instead,
fix recursion level in _read_config(),
don't create files when fuzzing,
fix memory leaks in deinit()
* src/stats.c: Don't create files when fuzzing
* src/wget.c: Remove set_exit_status()
* src/wget_main.h: Remove exit_status_t
* src/wget_options.h: Add exit_status_t
* tests/test-plugin.c: Fix expected exit codes from 1 to 2
This is for application fuzzing (namely code from src/).
The code in src/ had to be prepared in certain ways, but it
doesn't add significant overhead, even makes the code cleaner
in some ways. Several of these changes have already been committed
into the master branch.
* Makefile.am: Remove '<stdout>' special handling in coverage rules
* libwget/Makefile.am: Generate css_tokenizer.c with proper automake
macros, prepend include line using sed.
* .gitlab-ci.yml: Place fuzz coverage report in public area
* Makefile.am: Use 'make check' in fuzz directory
* README.md: Add link to fuzz coverage report
* fuzz/Makefile.am: Remove make target 'fuzz-coverage'
* fuzz/coverage.sh: Remove file
* Makefile.am: Add 'fuzz' directory to SUBDIRS,
tune 'fuzz-coverage' make target,
print info on how to view coverage report.
* configure.ac: Add AC_PROG_CXX for C++ code in fuzz/
* fuzz/Makefile: Removed
* fuzz/Makefile.am: New file
* fuzz/libwget_bar_fuzzer.cc: Fix memleak
* fuzz/main.c: Add code for unit testing
The OSS-Fuzz targets will now be taken for unit testing as well.
That is, they will be tested by 'make check' including our sanitizer
and valgrind checks. The fuzz tests will be run once with each test
corpus from the corresponding '.in' directory.
A 'make fuzz-coverage' generates a coverage report for running all
fuzz targets against their corpora.
* Makefile.am: Add target 'fuzz-coverage' to generate a coverage
report for fuzz targets.
* fuzz/Makefile: Add target 'fuzz-coverage'
* fuzz/coverage.sh: New script to call a fuzz target with each corpus.
* fuzz/libwget_metalink_parse_fuzzer.cc: Add check for max input length.
* fuzz/libwget_xml_parse_buffer_fuzzer.cc: Likewise
* fuzz/libwget_robots_parse_fuzzer.cc: Add comment
* fuzz/libwget_robots_parse_fuzzer.in/user_agent2: Amend for max coverage
* fuzz/main.c: selfmade __AFL_LOOP() returns 0 after first invocation.
* fuzz/libwget_memtohex.in: Rename to fuzz/libwget_memtohex_fuzzer.in
* fuzz/libwget_metalink_parse.in: Rename to fuzz/libwget_metalink_parse_fuzzer.in
* fuzz/libwget_robots_parse.in: Rename to fuzz/libwget_robots_parse_fuzzer.in
* fuzz/libwget_xml_parse_buffer.in: Rename to fuzz/libwget_xml_parse_buffer_fuzzer.in
* fuzz/libwget_xml_parse_buffer.options: Removed
* fuzz/libwget_metalink_parse_fuzzer.options: Removed
* Makefile.am: Add unit-tests
* cfg.mk: Make syntax-check work with unit-tests/
* configure.ac: Add unit-tests/Makefile
* tests/Makefile.am: Remove unit tests
* unit-tests/Makefile.am: New file with unit test rules
* unit-tests/*.c: Moved from tests/
* unit-tests/files: Moved from tests/
unit-tests/ is for library/API unit testing.
tests/ is for testing wget2 utility.
* .travis.sh: Rewrite for clarity and speed. Testing shows that valgrind
and ASan don't play well together. So split their executions.
* .travis.yml: Enable ccache for builds. Also try to update libc6 if
possible
* .travis.yml: Do not run on OSX with CC=gcc. This is because on OSX,
gcc is only a wrapper around clang.
* .travis_setup.sh: Cosmetic changes only
* Makefile.am (check-coverage): Explcicitly disable valgrind tests when
checking coverage
* tests/libtest.c (wget_test): Use a provided suppressions file for
valgrind. This is required since the valgrind version on Travis seems to
be out of sync with libc. This causes valgrind to report a memory bug
falsely.
* tests/valgrind-suppressions: New file with some valgrind suppressions.
Included suppressions are for a valgrind/libc bug and to suppress
reachable memory messages from within gnutls
* .travis.sh: Run `make check-coverage` when running with CC=gcc
* .travis.yml: Use the newer trusty tahir builds for updated packages
* .travis.yml: Refactor. Split out the OSX installation steps into
.travis_setup.sh
. .travis_setup.sh: Same
* Makefile.am: Add --ignore-errors source to lcov command line
* .gitignore: New entry examples/http_multi_get
* Makefile.am: Move subdir 'examples' before 'src'
* examples/Makefile.am: Add http_multi_get
* examples/http_get2.c (main): Adapt code to API changes
* examples/http_multi_get.c: New example to show async/parallel
requests
* examples/websequencediagram.c (main): Adapt code to API changes
* include/libwget.h.in: New functions wget_http_request_set_header_cb,
wget_http_request_set_body_cb, wget_http_request_set_int.
Changed params of wget_http_get_response.
Removed wget_http_get_response_func, wget_http_get_response_stream,
wget_http_get_response_fd.
Changed params of wget_http_get_response_cb.
Add members pending_requests, received_http2_responses,
pending_http2_requests to wget_http_connection_t.
Add member req to wget_http_response_t.
Add members header_callback, body_callback, header_user_data,
body_user_data, response_keepheader to wget_http_request_t.
New function types wget_http_header_callback_t and
wget_http_body_callback_t.
* libwget/http.c: Implementation of the changes in include/libwget.h.in.
* libwget/http_highlevel.c: Adapt code to API changes
* libwget/ssl_gnutls.c: Fixed defaults for SNI, OCSP and
ALPN.
Adapt code to API changes.
* src/options.c: Reflect changes in libwget/ssl_gnutls.c
* src/wget.c: Adapt code to API changes
