* README.md: Mark options
* docs/wget2_manual.md: Mark URLs
* libwget/bar.c: Fix doc for bar_init(),
add doxygen close tag at end of file
* libwget/buffer.c: Fix doc for wget_buffer_reset()
* libwget/console.c: Move unused tag to front of param
* libwget/hpkp.c: Fix documented params for wget_hpkp_new()
* libwget/iri.c: Fix doc for wget_iri_supported()
* libwget/net.c: Fix doc for wget_tcp_printf()
* libwget/utils.c: Move unused tag to front of param
* configure.ac: Suppress -Wunsuffixed-float-constants for gcc,
add -Wno-disabled-macro-expansion to clang flags
* examples/getstream.c: Removed 'unused' attributes from argc and argv.
* include/wget/wget.h: Fix slot to int for bar functions.
* libwget/bar.c: Fix slot to int for bar functions, cast to int,
change type of bitfield 'redraw' to unsigned.
* libwget/hsts.c (wget_hsts_new): Fix int to bitfield conversion.
* libwget/http.c: Cast printf %p format to (void *).
* src/wget.c: Cast printf %p format to (void *),
remove ACTION_DONE from enum actions.
* src/wget_host.h: Fix double declaration of host_remove_job()
* tests/libtest.c (wget_test_check_filesystem): Fix type of 'rc'.
* tests/test-idn-robots.c: Fix comma to semikolon at end of line.
* tests/test.c (test_strcasecmp_ascii): Fix type of loop variable.
* libwget/bar.c: Fix typos,
(wget_bar_set_slots): Init mem only when needed,
add comments describing Escape sequences
* src/bar.c (bar_update_slots): Take nslots as param
* src/wget.c: Move call to bar_update_slots() from worker to main thread
* src/wget_bar.h: Amend prototype of bar_update_slots()
* include/wget/wget.h: wget_get_screen_size() must not be pure
* libwget/bar.c (_bar_get_width): Leave one space right to avoid linebreaks
* libwget/utils.c (wget_human_readable): Align # of bytes
* src/bar.c (wget_get_screen_size): Fix return value
* include/wget/wget.h: New function wget_get_screen_size. Replaces old
wget_determine_screen_width
* libwget/bar.c (_bar_get_width): Use wget_get_screen_size instead of
wget_determine_screen_width
* libwget/utils.c: Replace function wget_determine_screen_width with a
more generic wget_get_screen_size.
* include/wget/wget.h: New function wget_bar_screen_resized, as an
indicator that the screen size may have changed. Also, wget_bar_update
may now modify the *bar parameter
* libwget/bar.c: Move detection code for screen size into a separate
function called _bar_get_width.
(wget_bar_update): Check if the screen width has changed and
reallocate more space if necessary
* src/wget.c: Install a new signal handler for SIGWINCH
* include/wget/wget.h: Declare new function wget_determine_screen_width.
Also change prototype of wget_bar_init to not accept a max_width
parameter
* src/utils.c: Move function determine_screen_width from here ...
* libwget/utils.c: ... to here.
* src/utils.{c,h}: Delete empty file
* src/Makefile.am: Remove source files utils.{c,h}
* src/bar.c: Move detection of screen width from here ...
* libwget/bar.c: ... to here
* tests/test.c: wget_bar_init() no longer takes a max_width parameter
* include/wget/wget.h (_wget_bar_ctx): Remove unused elements
expected_size and filename. Rename slotpos to _slotpos to reflect that
it should not be used by the client.
* include/wget/wget.h: Remove declaration for old API functions
bar_register and bar_deregister. Define declarations for
wget_bar_slot_begin, wget_bar_slot_register and
wget_bar_slot_deregister.
* libwget/bar.c (_bar_slot_status_t): Define new enum to store the
status of a slot of the progress bar.
(_bar_slot_t): Remove unused element last_ctx. Also, define slot
elements: filename, file_size, bytes_downloaded and status.
(wget_bar_slot_begin): Define new function. Invoked when the client
has started downloading a new file.
(wget_bar_slot_register): New function to register a new file in a
progress bar slot.
(wget_bar_deregister): Remove old function
(wget_bar_slot_deregister): New function, called after file has
finished downloading. Successfully or unsuccessfully.
(wget_bar_deregister): Remove old function
(_bar_set_progress): last_ctx is no longer used. Instead data is
stored within the slot struct itself
(_bar_update_slot): Same
(_bar_print_final): Same
(wget_bar_deinit): Free the filename stored in the slot
* src/bar.c: Edit to reflect the changes in libwget API
* src/wget.c (_get_header): Same
(http_send_request): Same
(http_receive_response): Same
* src/wget_bar.h: Same
The mutex stdout was meant to ensure that multiple routines don't write
to the screen concurrently. However, this mutex is not required, since
such syncronization should be handled by the client using the library.
All other library methods for the progress avoid any form
synchronization primitives for this reason
* include/libwget.h.in: Add new function wget_bar_set_slots
* libwget/bar.c (_wget_bar_st): Add new struct member max_slots to store
the maximum number of slots of the progress bar
(wget_bar_init): Initialize the memory for all the possible
progress_bars
(wget_bar_set_slots): New function to modify the number of progress
slots at runtime. Can only increase the number of slots, not decrease.
* src/bar.c(bar_init): Remove unneeded variable lf. bar_init() is
invoked before any threads are created, hence do not print the required
newlines. This is now done by wget_bar_set_slots
(bar_update_slots): Wget2 interface for libwget's wget_bar_set_slots
(_error_write): Fix the position of acquiring the mutex. Don't make
changes to the screen while some other progress bar function may be
running
* src/bar.h: New function bar_update_slots
* src/wget.c: Create global variable nthreads.
(downloader_thread): Update the number of progress bar slots
* src/wget.h: New global variable nthreads
Saving the max, cols and ratio to the slot structure was an optimization
used to prevent unneeded refreshes of the progress bar on slow
connections. However, with the asynchronous progress bar, this
optimization is no longer necessary as we refresh the progress bar far
fewer times already. Also, as other elements like the ETA are added to
the progress bar, these checks will only cause more problems as the ETA
would need to be updated even if no data has been downloaded in some
time.
* libwget/bar.c: Remove ratio, max, cur, cols and first from the
_wget_slot_st struct and eliminate all references to them.
* libwget/bar.c(_wget_bar_st): New struct members known_size and
unknown_size to denote the character with which the progress bar is
filled in these two cases
(wget_bar_init): Initialize the new struct members with the right data
(wget_bar_update): Choose and print the right character based on
whether the size is known or not.
(_bar_print_final): Same
(wget_bar_deinit): Free the struct members
* libwget/utils.c (wget_human_readable): Set acc to 1000 and print to 2
decimal places. This is similar to the settings used in Wget 1.x.
* libwget/bar.c (_bar_slot_t): Define element human_size to be as large
as _BAR_DOWNBYTES_SIZE.
* bootstrap.conf: Remove module 'human'
* include/libwget.h.in: Amend wget_human_readable()
* libwget/bar.c: Use new function wget_human_readable()
* libwget/utils.c: Replace wget_human_readable()
* src/wget.c: Use new function wget_human_readable()
* include/libwget.h.in: Add wget_human_readable_alloc()
* libwget/utils.c: Add wget_human_readable_alloc()
* libwget/bar.c: Use wget_human_readable_alloc()
* src/bar.c (bar_init): Set log function after call to wget_bar_init()
* src/wget.c (main): Use wget_human_readable() to print bottom line of bar
When the screen is scrolled or resized, we often need to redraw the
entire progress bar. However, when some threads have completed
downloading, this can cause those progress slots to become blank since
their contexts were deleted and hence lost. To prevent this, keep a copy
of the last context in the slot, so we can access it later for printing
the progress bar.
* libwget/bar.c (_bar_slot_t): New member, last_ctx
(wget_bar_deregister): Create a copy of the context just before we
deregister. This ensures that we copy the latest data for printing
(wget_bar_update): If no context is registered, then try printing the
last context available
(_bar_print_final): Make the method signature similar to
wget_bar_update for consistency
* bootstrap.conf: Add the human module and its dependency unistd
* include/libwget.h.in: Replace method signature for wget_human_readable
* libwget/bar.c (_bar_slot_t): Add a new struct memeber for storing the
human readable string that it printed on the progress bar
(wget_bar_init): Initialise the new human_size struct member
(wget_bar_update): Use the new wget_human_readable() method
(_bar_print_final): Same
(wget_bar_deinit): Free the allocated memory for each slot
* libwget/utils.c (wget_human_readable): Replace the old function with a
call to the human_readable() method from gnulib
When the progress bar is updated for the last time (whether exiting due
to error, or download complete), we may want to print information not
normally presented in the progress bar, like total time taken. To allow
this, separate out the printing of the last update of the progress bar
* include/libwget.h.in (_wget_bar_ctx): Remove unused members, final and
cond
* libwget/bar.c: Separate out the bar slot selection for reuse.
(_bar_print_final): New method for printing the last update of a
progress bar
* libwget/bar.c (wget_bar_init): Invoke wget_bar_deinit() to free up the
memory allocated to the various progress bar sections. Similarly, invoke
wget_bar_free() to free up the memory allocated to the progress bar
itself
Give the progress bar its own thread and let it update the entire
display asynchronously at a specified time interval. The existing
implementation refreshes the progress bar for each network packet
downloaded. Over multiple downloader threads and a high speed network
connection this can lead to far too many redrawings of the screen. Also,
each of the downloader threads will block while trying to acquire the
thread mutex because another thread just retrieved a packet. While I
haven't profiled it, it seems like there would be extremely high lock
contention in the existing implementation. Instead, with a separate
thread, we can update all the progress bar slots simultaneously at
regular intervals.
* bootstrap.conf: Include Gnulib module "ioctl"
* include/libwget.h.in: Define always_inline, flatten and deprecated
compiler attributes for both GCC and Clang
* include/libwget.h.in: Export new functions wget_human_readable,
wget_bar_register and wget_bar_deregister.
* include/libwget.h.in (wget_http_response_t): Add new element
"cur_downloaded" to struct. This element keeps a track of the raw
number of bytes downloaded in the response
(_wget_bar_ctx): Define new struct for storing the progress bar
context.
* libwget/bar.c: Fix display of downloaded ratio. Ensure it does not
exceed 100% by comparing the raw downloaded bytes instead of
uncompressed downloaded bytes.
Add support for printing the filename and downloaded bytes to the
progress bar.
* libwget/http.c (wget_http_response_cb): Store the raw number of
bytes downloaded from the network in the response data
* libwget/utils.c (wget_human_readable): New function to convert an
int value to a human readable string
* src/bar.c: Use the entire screen width instead of just 70 columns
for the progress bar.
Provide methods to register and deregister a progress bar
context
(wget_bar_update_thread): New method for the progress bar thread
to update the output periodically.
* src/bar.h: Same
* src/job.h: Move definition of struct DOWNLOADER from main.c to
here
* src/progress.h: Move definition of _body_callback_context from
main.c to here
* src/utils.c: Add new method to determine the width of the screen
* src/utils.h: Same
* src/wget.c: Use the new progress bar contexts. Update, register
and deregister them
(main): Deinit the progress bar only if it was enabled
* src/Makefile.am: Add new files, progress.h, utils.c, utils.h
* include/libwget.h.in (wget_bar_init, wget_bar_update,
wget_bar_print): Change types to int and off_t.
* libwget/bar.c (wget_bar_init, wget_bar_update,
wget_bar_print): Change types to int and off_t.
* src/wget.c (_prepare_file): Add prototype. Change type of rc to
size_t.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
