Magnus Hagander f9486b54f1 Don't generate http requests with two Host: headers ...

This would cause http 400 requests at the server, unsurprisingly.

2012-02-07 13:05:41 +01:00

..

crawler

Don't generate http requests with two Host: headers

2012-02-07 13:05:41 +01:00

sql

Weigh hits based on score when given in sitemap

2012-02-07 12:02:59 +01:00