235 Commits

Author SHA1 Message Date
0fe5f9282d New function wget_logger_is_active()
* include/wget/wget.h: Add prototype
* libwget/Makefile.am: Add logger.h
* libwget/log.c: Include logger.h
* libwget/logger.c: Include logger.h, add wget_logger_is_active()
* libwget/net.c: Use wget_logger_is_active()
* libwget/private.h: Remove struct _wget_logger_st
2017-01-23 16:55:52 +01:00
bfcd65c12b Use typedefs for function pointer arguments
* include/wget/wget.h: Add typedefs
* libwget/cookie.c: Use typedefs
* libwget/css.c: Likewise
* libwget/css_url.c: Likewise
* libwget/decompressor.c: Likewise
* libwget/hashmap.c: Likewise
* libwget/hsts.c: Likewise
* libwget/http.c: Likewise
* libwget/init.c: Likewise
* libwget/io.c: Likewise
* libwget/list.c: Likewise
* libwget/logger.c: Likewise
* libwget/metalink.c: Likewise
* libwget/net.c: Likewise
* libwget/netrc.c: Likewise
* libwget/ocsp.c: Likewise
* libwget/private.h: Likewise
* libwget/robots.c: Likewise
* libwget/stringmap.c: Likewise
* libwget/tls_session.c: Likewise
* libwget/vector.c: Likewise
* libwget/xml.c: Likewise
* src/blacklist.c: Likewise
* src/host.c: Likewise
* src/options.c: Likewise
* src/wget.c: Likewise
* tests/stringmap_perf.c: Likewise
* tests/test.c: Likewise
2017-01-23 14:43:17 +01:00
315c8f7516 base46: Use size_t as string length param 2017-01-20 16:23:02 +01:00
dfc4b53eae Add typedef for wget_vector_browse() callback function
* include/wget/wget.h: Add typedef wget_list_browse_cb_t
* libwget/list.c: Use wget_list_browse_cb_t
2017-01-19 17:11:58 +01:00
566e9f7c70 Add typedef for wget_update() callback functions
* include/wget/wget.h: Add typedef wget_update_cb_t
* libwget/cookie.c: Use wget_update_cb_t
* libwget/hsts.c: Likewise
* libwget/tls_session.c: Likewise
2017-01-19 17:11:58 +01:00
09b172dfc0 * libwget/io.c (wget_ready_2_transfer): Fix INF timeout for select() 2017-01-18 20:17:14 +01:00
5495dab56a Fallback to select() if poll() not available
* bootstrap.conf: Add module select
* libwget/io.c (wget_ready_2_transfer): Add fallback code
2017-01-18 11:15:24 +01:00
8248bfecd1 Add console color abstraction
* docs/Makefile.am: Add libwget-console.3
* include/wget/wget.h: Add console routines
* libwget/Makefile.am: Add libwget/console.c
* libwget/console.c: New file with console functions
* src/log.c: Add WIN32 and console function calls
2017-01-06 14:23:37 +01:00
1f39794852 Add function wget_truncate()
* bootstrap.conf: Ad gnulib module ftruncate
* include/wget/wget.h: Add wget_truncate
* libwget/io.c: Add wget_truncate
* src/job.c: Call wget_truncate instead of truncate

The truncate() function is not covered by gnulib.
wget_truncate() is a wrapper around ftruncate(), which is covered
by gnulib.
2017-01-06 12:14:40 +01:00
b9e7f48612 * libwget/cookie.c (_cookie_db_save): Fix ferror() check
Reported-by: Gisle Vanem
2017-01-06 10:36:08 +01:00
9dd781756b * libwget/Makefile.am: Add more libs to LDADD 2017-01-06 10:29:10 +01:00
042065d44f Let MSVC compile libwget/init.c (wget_global_init)
Reported-by: Gisle Vanem
2017-01-05 22:13:46 +01:00
0354295e03 Add network init function to bootstrap sockets on Windows
* bootstrap.conf: Add gnulib module 'sockets'
* examples/http_get2.c: Call wget_net_init()
* include/wget/wget.h: Add wget_net_init() and wget_net_deinit()
* libwget/init.c (wget_global_init): Call wget_net_init(),
  (wget_global_deinit): Call wget_net_deinit()
* libwget/net.c: Add wget_net_init() and wget_net_deinit()
2017-01-05 12:09:25 +01:00
43f30f309e Rename wget_str_[v]asprintf to wget_[v]aprintf
* include/wget/wget.h: Rename wget_str_[v]asprintf to wget_[v]aprintf
* libwget/http.c: Likewise
* libwget/iri.c: Likewise
* libwget/printf.c: Likewise
* libwget/vector.c: Likewise
* src/options.c: Likewise
* tests/test-metalink.c: Likewise
2016-12-21 11:54:36 +01:00
5f6b0b29a6 Adjust GnuTLS priorities
* libwget/ssl_gnutls.c (wget_ssl_init): Use system defaults for
  secure_protocol "auto".
  (wget_ssl_open): Set compatibility mode for "auto".
2016-12-21 11:44:23 +01:00
f240b0002a * libwget/cookie.c (wget_cookie_db_init): Fix argument of psl_latest() 2016-12-21 11:23:00 +01:00
40b8da056e Reduce redundant code by using strchrnul()
* bootstrap.conf: Add gnulib module strchrnul
* libwget/http.c (_parse_proxies): Remove redundant code
* libwget/ssl_gnutls.c (wget_ssl_open): Likewise
2016-12-16 15:41:58 +01:00
582bc2890a * libwget/thread.c: Return a value in dummy wget_thread_cancel()
Reported-by: Gisle Vanem
2016-12-15 11:22:22 +01:00
38f0905f2f Take advantage of libpsl 0.16+
* libwget/cookie.c (wget_cookie_db_init): Use psl_latest() if possible
2016-12-12 15:59:48 +01:00
82bd6fee89 * libwget/http.c: Ignore Transfer-Encoding for HTTP/2 requests 2016-12-02 16:41:25 +01:00
fe967f4817 Export symbols only from wget.h
* libwget/iri.c: Move exporting of symbol wget_iri_schemes from here
* include/wget/wget.h: To here
2016-11-16 11:16:58 +01:00
0117ec4ff5 Fix integer overflow in ratio calculation
* libwget/bar.c (_bar_update_slot): Fix integer overflow in ratio calculation

Fixes #125
2016-11-15 15:12:41 +01:00
37b732f95a * libwget/bar.c (_bar_set_progress): Typecast the right value to integer 2016-11-12 16:00:40 +01:00
7e207e1e1f * libwget/ip.c: Fix syntax error in docs 2016-11-11 16:32:11 +01:00
7803f3a01a * libwget/io.c: Use open() instead of creat()
open() is covered by gnulib, creat() not.
2016-11-09 19:27:34 +01:00
8366ead860 * libwget/thread.c: Add missing wget_thread_cancel() dummy 2016-11-08 22:34:03 +01:00
2a73b683d8 * libwget/net.c: Include sys/ioctl.h on Windows.
(_set_async): Use ioctl() instead of ioctlsocket().

Reported-by: Gisle Vanem
2016-11-08 21:49:09 +01:00
58d18ad60c New function to check for literal IPv4/IPv6 addresses
* bootstrap.conf: Add gnulib module 'inet_pton'
* docs/Makefile.am: Add man page libwget-ip.3
* wget/wget.h (struct wget_iri_st): New field 'is_ip_address'.
* libwget/Makefile.am: Add new file ip.c
* libwget/ip.c: New file defining new function wget_ip_is_family()
* libwget/iri.c (wget_iri_parse): Check for literal IP address
* src/wget.c: Do not add to HSTS if literal IP address
2016-11-08 16:22:08 +01:00
ed53ecbfa1 Use gnulib basename
* bootstrap.conf: Add gnulib module 'dirname'
* libwget/io.c: Use base_name() to get file basename
2016-11-08 12:41:12 +01:00
18dfa9f39e * libwget/xalloc.c (wget_realloc): Call OOM callback if size is 0 2016-11-07 16:07:26 +01:00
c270cda231 New inline function to compute length of base64 decoded string
* include/wget/wget.h (wget_base64_get_decoded_length): new function
 * libwget/tls_session.c (_tls_session_db_load): use
   wget_base64_get_decoded_length().
2016-11-06 17:10:18 +01:00
8b1a701c21 Use wget_strdup() instead of strdup()
* examples/getstream.c: Use wget_strdup().
* libwget/cookie.c: Likewise.
* libwget/encoding.c: Likewise.
* libwget/http.c: Likewise.
* libwget/iri.c: Likewise.
* libwget/netrc.c: Likewise.
* src/options.c: Likewise.
* src/wget.c: Likewise.
* tests/test.c: Likewise.
2016-11-03 09:58:09 +01:00
ac83eb4a98 * libwget/hashmap_old.c: Removed 2016-11-03 09:58:02 +01:00
b049eb6d3a Fix gcc/clang warnings from the new manywarnings feature
* configure.ac: Suppress -Wunsuffixed-float-constants for gcc,
  add -Wno-disabled-macro-expansion to clang flags
* examples/getstream.c: Removed 'unused' attributes from argc and argv.
* include/wget/wget.h: Fix slot to int for bar functions.
* libwget/bar.c: Fix slot to int for bar functions, cast to int,
  change type of bitfield 'redraw' to unsigned.
* libwget/hsts.c (wget_hsts_new): Fix int to bitfield conversion.
* libwget/http.c: Cast printf %p format to (void *).
* src/wget.c: Cast printf %p format to (void *),
  remove ACTION_DONE from enum actions.
* src/wget_host.h: Fix double declaration of host_remove_job()
* tests/libtest.c (wget_test_check_filesystem): Fix type of 'rc'.
* tests/test-idn-robots.c: Fix comma to semikolon at end of line.
* tests/test.c (test_strcasecmp_ascii): Fix type of loop variable.
2016-11-02 22:19:41 +01:00
cb5f9a230f Check for gcrypt.h
* configure.ac: Check for gcrypt.h
* libwget/hashfile.c: Check for HAVE_GCRYPT_H
2016-10-17 21:05:23 +02:00
fd46f870b1 Add gnulib digest hash fallback
* bootstrap.conf: Add md2, md5, sha1, sha256, sha512 modules
* libwget/hashfile.c: Add code branch to use gnulib functions if needed
2016-10-17 21:04:36 +02:00
b701a06495 * libwget/hashfile.c: Remove unused defines 2016-10-17 11:41:01 +02:00
5225421c31 Make --header replace wget's standard headers
* libwget/http.c (wget_http_create_request): Add 'Host' header
  to header list,
  (wget_http_send_request): Do not add 'Host' hard-coded,
  (wget_http_request_to_buffer): Special treatment of 'Content-Length'
* src/wget.c (http_create_request): Replace wget's HTTP headers by
  user-provided headers, except 'Cookie' headers which will be appended.
2016-10-14 12:51:12 +02:00
861385a6c8 Separate progress bar logic into library and application parts
* include/wget/wget.h: Remove struct _wget_bar_ctx and wget_bat_ctx_t,
  amend prototypes of wget_bar functions,
  new function wget_bar_write_line()
* libwget/bar.c: Bunch of cleanups and code rearrangements
* src/wget_progress.h: Remove
* src/Makefile.am: Remove src/wget_progress.h
* src/bar.c: Remove mutex usage,
  (_error_write) call wget_bar_write_line
* src/log.c: Remove redundant param checks
* src/wget.c: Add struct _body_callback_context,
  always call bar_slot_begin() on begin of new download,
  use downloader->id as progress slot
* src/wget_bar.h: Amend prototypes of bar_ functions
2016-10-13 16:39:59 +02:00
968e665973 Call bar_update_slot() from main thread
* libwget/bar.c: Fix typos,
  (wget_bar_set_slots): Init mem only when needed,
  add comments describing Escape sequences
* src/bar.c (bar_update_slots): Take nslots as param
* src/wget.c: Move call to bar_update_slots() from worker to main thread
* src/wget_bar.h: Amend prototype of bar_update_slots()
2016-10-12 17:09:09 +02:00
21099bcf7f * libwget/http.c: Fix update of downloaded bytes for progress bar 2016-10-11 15:56:54 +02:00
b908178f53 * libwget/ssl_gnutls.c: Print debug messages using debug_printf() 2016-10-11 15:55:35 +02:00
bc0fc2b28d * libwget/http.c: Support compression for HTTP/2 2016-10-10 15:44:55 +02:00
49ab9740e1 Fix progress bar details
* include/wget/wget.h: wget_get_screen_size() must not be pure
* libwget/bar.c (_bar_get_width): Leave one space right to avoid linebreaks
* libwget/utils.c (wget_human_readable): Align # of bytes
* src/bar.c (wget_get_screen_size): Fix return value
2016-10-05 11:49:45 +02:00
40340af300 Export a new function for determining screen dimensions
* include/wget/wget.h: New function wget_get_screen_size. Replaces old
wget_determine_screen_width
* libwget/bar.c (_bar_get_width): Use wget_get_screen_size instead of
wget_determine_screen_width
* libwget/utils.c: Replace function wget_determine_screen_width with a
more generic wget_get_screen_size.
2016-10-04 21:31:34 +02:00
354c8624d1 Handle SIGWINCH for progress bar
* include/wget/wget.h: New function wget_bar_screen_resized, as an
indicator that the screen size may have changed. Also, wget_bar_update
may now modify the *bar parameter
* libwget/bar.c: Move detection code for screen size into a separate
function called _bar_get_width.
  (wget_bar_update): Check if the screen width has changed and
reallocate more space if necessary
* src/wget.c: Install a new signal handler for SIGWINCH
2016-10-04 21:30:00 +02:00
e10cf1d1f3 Move window size detection to libwget
* include/wget/wget.h: Declare new function wget_determine_screen_width.
Also change prototype of wget_bar_init to not accept a max_width
parameter
* src/utils.c: Move function determine_screen_width from here ...
* libwget/utils.c: ... to here.
* src/utils.{c,h}: Delete empty file
* src/Makefile.am: Remove source files utils.{c,h}
* src/bar.c: Move detection of screen width from here ...
* libwget/bar.c: ... to here
* tests/test.c: wget_bar_init() no longer takes a max_width parameter
2016-10-04 20:29:30 +02:00
4cdffd2def Cleaner API for progress bar within libwget
* include/wget/wget.h (_wget_bar_ctx): Remove unused elements
expected_size and filename. Rename slotpos to _slotpos to reflect that
it should not be used by the client.
* include/wget/wget.h: Remove declaration for old API functions
bar_register and bar_deregister. Define declarations for
wget_bar_slot_begin, wget_bar_slot_register and
wget_bar_slot_deregister.
* libwget/bar.c (_bar_slot_status_t): Define new enum to store the
status of a slot of the progress bar.
  (_bar_slot_t): Remove unused element last_ctx. Also, define slot
  elements: filename, file_size, bytes_downloaded and status.
  (wget_bar_slot_begin): Define new function. Invoked when the client
  has started downloading a new file.
  (wget_bar_slot_register): New function to register a new file in a
  progress bar slot.
  (wget_bar_deregister): Remove old function
  (wget_bar_slot_deregister): New function, called after file has
  finished downloading. Successfully or unsuccessfully.
  (wget_bar_deregister): Remove old function
  (_bar_set_progress): last_ctx is no longer used. Instead data is
  stored within the slot struct itself
  (_bar_update_slot): Same
  (_bar_print_final): Same
  (wget_bar_deinit): Free the filename stored in the slot
* src/bar.c: Edit to reflect the changes in libwget API
* src/wget.c (_get_header): Same
  (http_send_request): Same
  (http_receive_response): Same
* src/wget_bar.h: Same
2016-10-04 20:23:56 +02:00
6a115f1b1e * libwget/bar.c: Remove mutex stdout
The mutex stdout was meant to ensure that multiple routines don't write
to the screen concurrently. However, this mutex is not required, since
such syncronization should be handled by the client using the library.
All other library methods for the progress avoid any form
synchronization primitives for this reason
2016-10-03 13:12:31 +02:00
7be16cf925 * libwget/http.c: Remove unused struct _body_callback_context 2016-10-03 13:04:22 +02:00