Tim Rühsen 2ec7c7e290 Deduplicate code for creating robots.txt jobs ...

* src/wget.c (queue_url_from_local): Remove duplicated code,
  (queue_url_from_remote): Likewise,
  (set_file_metadata): Amend calling wget_iri_get_connection_part()
* include/wget/wget.h (wget_iri_parse_base): Add 'const' to 'base' param,
  (wget_iri_get_connection_part): Add 'const' to 'iri' param,
  add second param 'buf'
* fuzz/libwget_iri_fuzzer.c (test): Amend calling wget_iri_get_connection_part()
* libwget/iri.c (wget_iri_get_connection_part): Add param 'buf',
  add 'const' to 'iri' param, amend docs,
  (wget_iri_relative_to_abs): Add 'const' to 'base' param,
  amend calling wget_iri_get_connection_part(),
  (wget_iri_parse_base): Add 'const' to 'iri' param
* src/host.c (host_add_robotstxt_job): Add code removed in wget.c
* src/wget_host.h: Don't include wget_blacklist.h,
  amend signature of host_add_robotstxt_job()
* src/wget_job.h (struct JOB): Add 'const' to 'blacklist_entry'

2019-10-26 18:48:33 +02:00

..

wget

Deduplicate code for creating robots.txt jobs

2019-10-26 18:48:33 +02:00

Makefile.am

…