There is no need to browse into the repositories on the main website.
It's still possible to browse there directly on the ftp servers of course,
for those that need to debug a repository install and things like that.
Previously this had to be rsynced outside of the website. By allowing the
upload here, and automatically purging the data from varnish, we will reach
"almost instant" updates of the ftp site structure on the web.
on the ftp server, instead of crawling the directoreis directly. This
removes the requirement to sync almost 10Gb worth of ftp site onto the
web server...
The pickle file for this is currently around 1Mb, so it's not a huge
burden on the server. If it grows larger in the future, we may want to
re-think this and split it up, or put it in a database format or something
like that.
