Ubuntu: Extract all files after wget download


I'm using wget to download a list of files from a URL, here is my command:

wget --user='username' --password='mypassword' -i  url.txt  

This downloads a list of .gz files

What I need to do is:

  1. Before downloading, delete all files in directory
  2. After downloading, unzip all files and delete zipped ones

Is there a command or list of commands which might accomplish this which I can insert into a cronjob? I need to run this once a day.



Let's try a one-liner :

rm -rfi * && wget -q -O - --user='username' --password='mypassword' -i  url.txt | wget - && find . -name "*.zip" -exec "unzip && rm" {} \;  

Don't hesitate to edit/adapt : this could be really wrong as I'm not always very at ease with the - pseudo-descriptor and how each application handles it.

Some details

Clear current directory.

rm -rfi *  

Get the list, but instead of sending the contents to a file, redirect it to stdout (using -q, -O and the pseudo-descriptor -).

wget -q -O - --user='username' --password='mypassword' -i  url.txt   

Download files names of which are being sent from the previous wget call. Again, - makes refers to the standard input, which is also the previous command's output.

wget -  

Find all zip files in the directory, and execute unzip and rm on each of them.

find . -name "*.zip" -exec "unzip && rm" {} \;  


This is more of a *nix question, than Ubuntu-centric.

In general you'd write a script (using the bash language, or perhaps Python) that handles the task you describe. One line of the script would be your command, above... the rest would handle the inspection of the directory, deletion, unzipping etc.

Then you'd just use cron (google up crontab) to set the script to run at specific times.

Note:If u also have question or solution just comment us below or mail us on toontricks1994@gmail.com
Next Post »