Previously I've written the following backup scripts:
RoboBackup - a simple script using RoboCopy, it had folder recycling.
RoboBackup7z - similar but it used a combination of RoboCopy and 7-zip to backup files.
In this article we'll take it a step further, combining backup recycling and archiving features. Also, adding the ability to backup from multiple sources to multiple destinations. This new script is called RoboBackup2:
Source
Here's the source code for the script:
#cs ----------------------------------------------------------------------------
RoboBackup version 2
AutoIt Version: 3.3.10.2
Author: Michael Gerrard, http://mgxp.blogspot.com
Script Function:
Create multiple backups of the same data using the RoboCopy mirror function
#ce ----------------------------------------------------------------------------
; Set variables
$title = "RoboBackup"
$ini = @ScriptDir & "\RoboBackup.ini"
$retry = IniRead($ini, "settings", "retry", "1") ;number of retries
$wait = IniRead($ini, "settings", "wait", "2") ;sec to wait
$exclude = IniRead($ini, "settings", "exclude", " ") ;files to exclude
$attrib = IniRead($ini, "settings", "attrib", "/A-:SH /A+:R") ;attributes
$parameters = IniRead($ini, "settings", "parameters", " ") ;more parameters
$backups = IniRead($ini, "settings", "backups", "3") ;number of backups
$count = IniRead($ini, "settings", "count", "1") ;count of backups
$destPath = IniRead($ini, "destination", $count, @ScriptDir)
$dest = $destPath & "\Backup" & $count ;Backup1, Backup2, etc
$log = @ScriptDir & "\Backup" & $count & "_Log.txt"
$7za = @ScriptDir & "\7za.exe" ;the 7-zip command line exe
$archPass = IniRead($ini, "archive", "password", Random(10000000, 99999999, 1))
$archPassPath = IniRead($ini, "archive", "list-path", @ScriptDir)
$archPassFile = $archPassPath & "\ArchiveList.ini"
$archSize = IniRead($ini, "archive", "volume-size", "300m") ;300MB default
$archPath = IniRead($ini, "archive", "path", @ScriptDir) ;archive path
$archLast = IniRead($ini, "archive", "last", "0000-00") ;last archive?
$archNow = @YEAR & "-" & @MON ;year and month now, used as the file name
$arch = $archPath & "\Archive\" & $archNow ;path and filename
$archZip = IniRead($ini, "archive", "zip", "no")
If $archZip = "yes" Then
$zip = "-tzip" ;set to zip
Else
$zip = "-t7z" ;if not set, use the native 7z format
EndIf
; Read the Source folders into an array
$aSource = IniReadSection($ini, "source")
If @error > 0 Then ;if there are no source folders, add default ones
IniWrite($ini, "source", "1", @UserProfileDir & "\Documents")
IniWrite($ini, "source", "2", @UserProfileDir & "\Pictures")
IniWrite($ini, "source", "3", @UserProfileDir & "\Music")
IniWrite($ini, "source", "4", @UserProfileDir & "\Videos")
IniWrite($ini, "source", "5", @DesktopDir)
IniWrite($ini, "source", "6", @FavoritesDir)
MsgBox(16, $title, _
"ERROR: No source folders have been specified in the INI." & @CR & @CR & _
"The default source folders have been added to the INI." & @CR & @CR & _
"Click OK to exit the backup:" & @CR & _
" - The ini will open automatically" & @CR & _
" - Review/edit the source folders" & @CR & _
" - Run the backup again")
Run("Notepad " & $ini)
Exit
EndIf
; Check if home folder has a space in it
If StringInStr(@ScriptDir, " ") > 0 Then
MsgBox(16, $title, "ERROR: " & $title & " cannot run!" & @CR & @CR & _
"The home folder must *not* contain spaces." & @CR & @ScriptDir)
Exit
EndIf
; Check if the destination path is ok
If DirCreate($destPath & "\T_E_S_T") = 0 Then
$destOK = MsgBox(21, $title, "ERROR: " & $destPath & " NOT FOUND!" & @CR & @CR & _
"If the destination is a USB drive please make sure it is plugged in" & @CR & _
"and click Retry. This window will timeout in 2 minutes.", 120)
; Retry
If $destOK = 4 Then
If DirCreate($destPath & "\T_E_S_T") = 0 Then
MsgBox(16, $title, "ERROR: " & $destPath & " still NOT FOUND!")
Exit
EndIf
Else ;if Cancel...
Exit
EndIf
EndIf
; If successful remove the test folder and continue
DirRemove($destPath & "\T_E_S_T")
; Backup using the Destination paths
If FileExists($log) Then FileDelete($log) ;delete the existing log
For $s = 1 to $aSource[0][0] ;read each source folder one by one
$source = '"' & $aSource[$s][1] & '"' ;format the source folder
; Run RoboCopy
$roboError = RunWait("ROBOCOPY " & $source & " " & $dest & "\" & $s & _
" /MIR /R:" & $retry & " /W:" & $wait & " /LOG+:" & $log & " /TEE" & _
" /XF desktop.ini thumbs.db *.tmp " & $exclude & " " & _
$attrib & " " & $parameters, @ScriptDir, @SW_MINIMIZE)
Next
; If RoboCopy returned a serious error, give a warning
If $roboError > 7 Then
MsgBox(16, $title, "ERROR: Please review the log file for Backup" & $count)
Run("notepad " & $log) ;Display the log file
Else
; Test that the archive folder can be written to
$testOK = DirCreate($archPath & "\T_E_S_T")
DirRemove($archPath & "\T_E_S_T")
; If the back was successful then check if it's time to Archive
If $archLast <> $archNow AND FileExists($7za) AND $testOK = 1 Then
DirCreate($archPath) ;make sure the archive folder exists
; Create an archive file for each source folder
For $s = 1 to $aSource[0][0]
RunWait($7za & " a -r " & $zip & " -v" & $archSize & " -w -y -p" _
& $archPass & " " & $arch & "(" & $s & ") " & _
$dest & "\" & $s & "\*.*", @ScriptDir, @SW_MINIMIZE)
Next
; Update the ini file and display the backup and archive message
IniWrite($ini, "archive", "last", $archNow)
IniWrite($archPassFile, "archives", $archNow, $archPass)
MsgBox(64, $title, "Backup" & $count & " OK" & @CR & _
"Archive created: " & $archNow)
Else ;no error and no archive
MsgBox(64, $title, "Backup" & $count & " OK")
EndIf
EndIf
If $count >= $backups Then $count = 0 ;reset the counter
IniWrite($ini, "settings", "count", $count + 1) ;update the count ini
Exit
Copy and paste the above script into a text editor (Notepad for example). Save as RoboBackup.au3
Compile the RoboBackup.exe - to do this you will need AutoIt installed. You can download AutoIt from here: https://www.autoitscript.com
For the archive to work you'll also need the 7za.exe. You can download it free of charge, full details can be found here: https://mgxp.blogspot.com/2016/04/7-zip-7za-command-line-zip-tool.html
Quick Start Tutorial
RoboBackup2 can run from almost anywhere, directly from a USB drive or from your local hard disk drive. The only restriction is that it cannot be on the Desktop or in another location where there are spaces in the folder path.
In our example we've created a folder D:\RoboBackup2, this will be our 'home folder' and we have put our compiled script, RoboBackup.exe in that folder. Double click RoboBackup.exe and (by default) the following folders (files and sub-folders included)...
C:\Users\<username>\Documents
C:\Users\<username>\Pictures
C:\Users\<username>\Music
C:\Users\<username>\Videos
C:\Users\<username>\Desktop
C:\Users\<username>\Favorites
...will be copied to D:\RoboBackup2\Backup1, etc...
 |
Once it has finished, RoboBackup2 will display a message saying "Backup1" has finished. Now you'll have a new folder D:\RoboBackup2\Backup1. This is where the backed up files are stored. There are sub-folders below that are numbered 1-6. They represent the six default source folders that are backed up. 1=Documents, 2=Pictures, etc (see above list). Also, in the D:\RoboBackup2 folder you'll find a log file Backup1_Log.txt - this is a log file that explains in detail which files were copied, if there were errors and more.
Run RoboBackup.exe again, a second folder will be created called Backup2.
Run RoboBackup.exe a third time and Backup3 will be created.
Run RoboBackup.exe a forth time and Backup1 will be updated. Only those files that have been changed on C: will be copied. Any files that were deleted in the source folder will be deleted from the Backup1 folder.
The above behaviour is called recycling, this is explained in more detail later in this article.
If 7za.exe is present in the 'home' folder (D:\RoboBackup2 in our example) a password protected archive will be created once a month - more details below.
Many of the defaults can be changed using the RoboBackup.ini file. The following explains how you may configure RoboBackup2 to meet your needs.
RoboBackup.ini
How RoboBackup behaves can be configured using the RoboBackup.ini file. It is automatically created the first time you run the script or you can create it yourself from the beginning. Here is a copy of the ini with the default settings:
[source]
1=C:\Users\<username>\Documents
2=C:\Users\<username>\Pictures
3=C:\Users\<username>\Music
4=C:\Users\<username>\Videos
5=C:\Users\<username>\Desktop
6=C:\Users\<username>\Favorites
[settings]
count=1
backups=3
retry=1
wait=2
exclude=
attrib=/A-:SH /A+:R
parameters=
[archive]
path=
list-path=
volume-size=300m
last=
password=
zip=
[destination]
In the Source section you may continue adding as many folders as you wish. All folders and their respective sub-folders will be included in the backup. When backed up the folders appear as their respective numbers. For the above example you would have:
Backup1\1
Backup1\2
Settings
In the RoboBackup.ini there is a section titled settings. Let's explain them one-by-one:
count=1
This increments every time you run the script. It is how the script keeps track of which backup to make. Normally leave this alone.
backups=3
The number of backups. The default is 3 meaning that folders backup1, backup2 and backup3 will be created after running the script three times. If you set this to 4 then four backups will be made, if you set it to 5, five will be made, etc.
retry=1
This is the number of times Robocopy will try to copy the file. For example, if a file is locked for editing then Robocopy will not be able to copy it. With retry=1 it will try one more time. After that it will give up and the log will show this file was not backed up. Normally retry=1 is fine but if you are backing up files from a network shared drive you might consider changing this to retry=3 or more. However, the more retries, the slower the backup as it takes time to try again - see the next wait=.
wait=2
This is related to retry=x. The number of seconds to wait before retrying. For example, if you use retry=2 and wait=30 then it will take an extra minute until moving to the next file. This functionality is directly from the Robocopy command, please have a look on the web for more information.
exclude=
By default (exclude=) the following files are not included in the backup:
desktop.ini
thumbs.db
*.tmp
To add to this list and exclude more files use the exclude= parameter. You can specify multiple files by putting a space between each. For example:
exclude=*.mov *.mp4
The above would exclude all files ending in mov and mp4. This is in addition to the hard-coded desktop.ini, thumbs.db and tmp files.
attrib=/A-:SH /A+:R
By default files backuped up have the System and Hidden attributes removed (it's good to see the files you have backed up). The backup up files in the destination are set to Read-only. The idea is to stop backed up files being accidentally deleted or changed. However, in practice it is very easy to delete or change a read-only file so this is not very serious protection. The default configuation is included by default for you to modify as you wish. If you remove it and leave attrib= then no attribute changes would be made.
parameters=
Robocopy has lots of parameters and if you'd like to add something specific you may, using parameters=. For example, perhaps you want to restrict files of a certain size being included in the backup? You could do this by adding:
parameters=/MAX:1073741824
The number is bytes, the above would stop incidvidual files of 1GB and over from being included in the backup.
If you are interested in knowning more about the functionality and further parameters you could use, look up Robocopy on the web or look at its help from the command line (robocopy /?).
Backup Recycling
Every time you run the backup, if a new folder was created you would soon run out of storage space. This is especially true if you backup every day. Also, it would take a long time to backup all the files each time. It would make more sense if you would replace your oldest backup at some point. If only the changed files were replaced, this would speed up the backup time considerably. Therefore the script works like this:
It creates one backup each time it runs. By default three backups are created, when the script runs for a fourth time, the first backup is recycled. Here's an example:
- The first time the script is run Backup1 is created.
- The second time and Backup2 is created.
- The third time and Backup3 is created.
- The forth time and Backup1 is updated.
- The fifth time and Backup2 is updated.
- The sixth time and Backup3 is updated.
- The seventh time and Backup1 is updated.
- It continues like this...
The number of backups can be changed from 3 to any number you wish. In the ini file, under [settings] add the line backups=X where X is the number of backups you would like. For example, set it to 7 and run the script every day to have a daily backup (assuming the script is run only once a day).
Destination
Each backup folder is created in the script folder by default. However, you can decide where each backup folder is stored. In the ini there is a [destination] section. Add the destination folders you need. This means you could spread your backups over a number of different locations/drives.
For example, you might have:
[destination]
2=D:
3=K:
This would mean that backup 1 would be stored in the home folder (where the RoboBackup2 script is). Backup 2 would be saved to D: and backup 3 to drive K:. Dispersing the backups like this means if one backup drive fails, another will be fine. You could also put one or more of these backup drives in a safe place, perhaps off-site.
Archive
This is an optional feature. If the 7za.exe file is present in the home folder, once a month the backup runs as usual but after the backup is complete it is compressed into archive 7z file(s). An archive is created for each source folder in the backup. For example:
Backup1\1
Backup1\2
The above would archive as (assuming the date was October 2017):
2017-10(1).7z
2017-10(2).7z
In the above example two archive files are created, one for each source folder backed up. Also, each archive file is split into volumes. By default 300MB volumes. This means that if a total backup was 700MB, one folder was 500MB and contained 200MB, you would have the following archive files:
2017-10(1).7z.001
2017-10(1).7z.002
2017-10(2).7z.001
The archive files are encrypted (AES-256) with a password. The password is an eight digit random number. Every time the archive runs the ArchiveList.ini file is updated with the name (year-month) of the archive and the password. The ArchiveList.ini is created in the home folder by default but you can also specify where it is stored using the list-path= parameter. This is important because it is best to keep the passwords separate from where you store the archive. But don't lose the ArchiveList.ini file - cracking AES-256 encryption is close to impossible! Archive files are always encrypted.
If you don't need such security, you can specify a generic password with the password= parameter. For exmaple you could put password=12345
By default the native 7-zip format (.7z) is used for the archive files. If you would like to use the zip format, add the following to the [archive] section of the ini:
zip=yes
(it is case sensitive)
If you do not wish to use the archive feature you can disable it altogether. To do this simply delete the 7za.exe file. If the script does not see the 7za.exe it will not attempt to create an archive.
By default the archive file is created in a folder called "Archive" where the script is (and Backup folders). It is possible to specify a separate destination path for the archive. Here's an example:
[archive]
path=K:
After it runs for the first time you'll have a K:\Archive folder and all archive files (7z or Zip) will be stored here.
The first time the script runs it will check to see if this path exists. If it does not, the archive will not be created, the archive process will be skipped. No error message appears. The next time the script is run, again it will look for this path, if not found, it will skip the process.
The idea is that the archive backup drive (USB removable hard disk, flash drive, etc) will not be connected to the computer all the time. Therefore, when it is plugged in, possibly just once a month, when the script runs the archive will be made. This is why no error message appears when the path is not found. It is assumed that is your choice, that you will plug the archive drive in later.
Error
Once RoboBackup has finished it will display a message stating that the backup completed. If there is an error, perhaps there is not enough space on the destination drive, the following message will appear:
If you see a message like this, review the corresponding log file, it will explain what happened.
Example Backup Scenario 1
In the tutorial at the beginning of this article we described a simple backup solution where files from C: would be copied to backup folders on D:. This was assuming D: was a local drive. However, it would be safer if it were a removable drive, you would be able to put it in a safe place after the backup had finished.
Put the RoboBackup2 home folder on drive G: (the USB flash drive in the above example). Run RoboBackup.exe and it will backup from the C: to G:, creating backup folders such as:
G:\RoboBackup2\Backup1
G:\RoboBackup2\Backup2
G:\RoboBackup2\Backup3
Example Backup Scenario 2
This is very similar to the first scenario but we have two USB flash drives:
[destination]
3=H:
After you've run RoboBackup at least three times you will have the following backups:
G:\RoboBackup2\Backup1
G:\RoboBackup2\Backup2
H:\RoboBackup2\Backup3
This would mean that every third backup would be stored on the second USB drive. This could be useful for two reasons; 1) if one USB drive dies you still have the other, 2) if three backups would not fit on one USB drive.
Of course you could set this up in any way you wish.You could store the RoboBackup2 home folder on the D: drive and specify the Destinations for backups 2 and 3 to be on the USB drives. You'd have three backups on three separate devices. Or use three USB drives... or...
IMPORTANT: If the USB drive is not present when the backup runs, the backup will fail. Look in the log file for an explanation.
Example Backup Scenario 3
In this scenario we'll see the real power of RoboBackup2:
The two USB drives would be plugged in all the time.
The RoboBackup2 home folder would be stored on G: (the first USB drive).
In the Settings section of the ini file the number of backups would be limited to two:
[settings]
backups=2
In the Destination section H: would be specified for the second backup:
[destination]
2=H:
The above would spread the backups over the two USB drives, this would mean you'd have two backup copies available at all times. If one USB drive died, the other would be available, so you'd have at least a backup from yesterday or the previous day at the worst.
In addition in the ini the Archive section should look like this:
[archive]
path=M:
The drive you use for the archive should have plenty of disk space, this is why in the diagram it's shown as an external USB hard disk drive. One archive would be created once a month, the first time during the month that the drive was plugged in. This means that if you forget to plug drive M: in, it doesn't matter. Just plug it in the next time RoboBackup2 is run and it'll create the archive.
An advantage of using the archive feature is that you should unplug that drive and store it somewhere else, even off-site. Only plug it in once a month. This is a good idea because if a virus hits your computer it might infect any connected drives. Having at least something disconnected is always a good idea and this could be a good system for you. It still requires some discipline from your side of course!
Don't forget that when the archive runs the ArchiveList.ini is created in the home folder. Keep that file safe as all archive files are password protected, the random password is stored in the ArchiveList.ini file.
Conclusion
I've been using this script for real for well over a year. It works well. It's meant some peace of mind. I know my backups are there and accessible. I know that every month there's an additional copy (the archive) that I can always refer to. Indeed having that monthly archive is great in case a month or so down the line I realise I should not have deleted a file. I can still get it back.
This script isn't for everyone of course. But there is something nice about having a script rather than a backup program with a user interface. With a script like this you set it up once and just let it do its thing. Don't forget you could schedule the RoboBackup.exe to run at a certain time every day using the Windows Task Scheduler.
I hope you've found this script interesting. Feel free to use it how you wish, of course a little credit would be nice. Maybe you found a better way to code something? Maybe you expanded the functionality? Please reply in the comments below!
Disclaimer
My script works fine for me but I accept no liability for anything here. I'm providing this script source code and this article for you to work with as you wish. Use at your own risk! If you lost data from a backup not working, again this is nothing to do with me, all of what I've explained here is just my opinion, my advice.
No comments:
Post a Comment