[sdiy] archiving info

[Dave Manley]

Andre Majorel wrote:
> Some variation on
>
>   wget -l 10 --limit-rate 10k -N -np -r -w 1 $url
>
> If you mirror the same site more than once, always do it from the
> same directory on your disk or you will download everything again
> (as opposed to just the bits that have changed since last time).
> First read the man page. Then test it on a small subtree of the
> site. When in doubt, always ask for advice.
>   
I had a posting all ready to send about wget, but decided not to send it.

wget is a great program, but if you are inexperienced in its use, please 
watch it carefully after you start it and make sure it hasn't started 
retrieving unintended files or that it is storing the files locally in a 
way you don't prefer (there are link rewriting options, directory 
options, etc).  As Andre suggests try it on a small subtree and make 
sure it is doing the right thing, otherwise you may find yourself 
wasting someone else's bandwidth multiple times and have a bunch of 
local files that aren't as useful as you expected.

-Dave