July 19, 2003

Automated backup solution

Whilst doing the washing up this afternoon I realised it was going to be possible to automate the backup of my webspace so that I had a complete backup of the webpages, mysql databases and the various alias options that are available in the backup web page under my domains control panel.

I downloaded and extracted the latest (beta) version of wget and also the corresponding ssllibs.

Once these two zip files were extracted to a directory I then created a batch file (w.bat) which does the following:-
Creates and changes into a temp directory.
Downloads all the files found from the main backup page location, which in my case is http://absoblogginlutely.net:2082/frontend/Xskin/backup/index.html
It then moves the directories named getsomethingbackup into a directory named with the current date and time and finally deletes all the files in the temp directory.

Note that at present this file should work ok with WindowsXP (and probably win2k) and should work with sites hosted on hasweb. If you are hosting somewhere else, then as long as your cpanel contains an interface to a backup routine you can use the same method to backup.
Note you are entirely on your own with support on this - I am not responsible for deleting your entire hard disk or any other nightmares you get
Feedback should be sent via the comment functionality.

The only annoying thing about this is it takes ages to test as my main backupfile is over 30mb, although I am getting a transfer rate of 57K/s

Posted by Andy at July 19, 2003 3:28 PM
Comments

w.zip is updated because hasweb changed their control panel skin. for those of you who don't use the same skin as hasweb, just change the line http://%domain%:2082/frontend/x/backup/index.html to the correct path for your backup screen in control panel.

Posted by: Andy at August 1, 2003 12:23 PM

http://absoblogginlutely.net/cpanelbackup/w-1.3.zip is the latest version updated to include backups of crontab jobs
Main page for this program is at http://absoblogginlutely.net/cpanelbackup for further documentation and a mailing list at http://absoblogginlutely.net/mailman/listinfo/cpanel.backup_absoblogginlutely.net so you can get updates to this program

Posted by: Me at October 11, 2003 12:57 PM

I was wondering if there is a way to exclude directories with this backup?

Posted by: ryan at January 20, 2004 7:05 PM

nope, as I get the cpanel zipped version of all files in the account directory - now if you can get cpanel to ignore a directory...... (I must admit it would be nice to ignore my images directory sometimes!)

Posted by: Me at January 20, 2004 8:19 PM

Very Helpful, got this set up easily and makes me sleep a little easier now. Made 2 different versions of the w.bat file, one to download the website and one to just download the sql files (which I schedule more often then the main download, of course). For anyone who wants to know:

To just do SQL directories, I added:
-X/getbackup,/getaliasbackup,/getfilterbackup

to the 'get full backups' line.

Also, to save about 30k per download, I added this to that same line:
-Rgif,ico,txt,css

Thanks again!

Posted by: Brian at April 26, 2004 9:06 AM

in w.bat, exactly where do you put the user, password and domain? It says after the name of the batch file, but woulodn;t it go in the folling lines?
set user=%1
set pass=%2
set domain=%3

ie change to:
set user=theusername
set pass=thepassword
set domain=thesite.com

Also,

where someone has said "To just do SQL directories, I added:
-X/getbackup,/getaliasbackup,/getfilterbackup
to the 'get full backups' line."


could you please post the exact line after the change?


Thanks!

Posted by: andrew at May 20, 2004 1:02 AM

Andrew, no the passwords are passed as commandline parameters. This way I don't have to get you to edit the batch file and have the possibility of you accidentally incorrectly editing it. if you want to hard code your own password etc in, then by all means replace those three lines as you suggested.

The full backup line would become
..\wget -r -l 1 --http-user=%user% --http-passwd=%pass% http://%domain%:2082/frontend/x/backup/index.html -X/getbackup,/getaliasbackup,/getfilterbackup

Posted by: me at May 20, 2004 7:35 AM

Thanks for the info!

One more question in regard to this line in the batch file:

rd /q /s %domain%_2082

what exactly is this line doing? removing a direcory?

Posted by: Andrew at May 21, 2004 8:17 AM

yes - deleting the directory absoblogginlutely.net_2082 which is created when you download (and contains data I don't actually need such as the images on the control panel page)

Posted by: me at May 21, 2004 10:03 PM

For ppl that wish to find out more about
Website Mirroring With wget,

or to obtain wget 1.9.1 and ssllibs 0.97,

this is a good read:

http://www.devarticles.com/c/a/Web-Services/Website-Mirroring-With-wget/

Posted by: Andrew at May 25, 2004 7:43 AM

I have this working to an extent that all files are downloaded

When it tries to move the backup files from the temp dir I get an error 'system cannot find specified file'

Any ideas appreciated

Posted by: JT at July 12, 2004 12:24 PM

Found on my XP system using the latest wget

for /d %%a in (%domain%_2082\*backup) do move %%a ..\%finalbackupdir%

need to change the _ to a +

Posted by: JT at July 13, 2004 12:50 AM

Have you tried the BOS software?
www.bos.co.il

Posted by: Meow at July 26, 2004 8:50 AM

hi..
is it possible to backup all the sites using cpanel's root password

Posted by: harry at May 1, 2005 6:30 AM

I updated this script after it broke at a customers site due to a slightly different page layout. I discovered that the script was also retrieving the logout link, logging out wget and therefore subsequent retrievals would not work.
In the process of debugging this script I also made the page slightly more robust in that it ignores robots.txt and uses a fake browser agent, and friendlier to the download agent in that it waits 2 seconds between downloads.
Use the following line in w.bat
wget -w 2 -r -l 1 -erobots=off -U "Mozilla/5.0 (compatible; Konqueror/3.2; Linux)" --http-user=%user% --http-passwd=%pass% -Rgif,ico,txt,css,html --exclude-directories /logout http://%domain%:2082/frontend/x/backup/index.html

This should all be on one line.

Posted by: Andy at March 16, 2007 9:32 AM