shellwget

Using wget to recursively fetch a directory with arbitrary files in it


I have a web directory where I store some config files. I'd like to use wget to pull those files down and maintain their current structure. For instance, the remote directory looks like:

http://mysite.com/configs/.vim/

.vim holds multiple files and directories. I want to replicate that on the client using wget. Can't seem to find the right combo of wget flags to get this done. Any ideas?


Solution

  • You have to pass the -np/--no-parent option to wget (in addition to -r/--recursive, of course), otherwise it will follow the link in the directory index on my site to the parent directory. So the command would look like this:

    wget --recursive --no-parent http://example.com/configs/.vim/
    

    To avoid downloading the auto-generated index.html files, use the -R/--reject option:

    wget -r -np -R "index.html*" http://example.com/configs/.vim/