A simple script to use either robocopy or xcopy to backup files

Under various circumstances, I've found it useful to cobble together a script to do a sync backup across the network from one Windows server to another. Usually this is for files only and is for either a mirror or a daily, full backup of data. Obviously there are some great backup tools available that make something like this largely unnecessary, however, this is quick, simple and gives you an email output of what has happened. The first example below uses robocopy (Robust Copy) which is a very nice bit of kit indeed. It's a bit more useful than xcopy and handles larger numbers of files better. Don't get me wrong, I love xcopy, but it has it's limitations. I use rsync a lot on Linux servers and robocopy gives me many similar options for how I want to handle files.

The destination directory could be anything - another folder on the same PC, a removable disk, a mapped share or even a straight UNC path e.g. \\server\share - flexibility is the key for this script, once the basic variables are right and you've decided to use robocopy or xcopy then off you go.

So to the script:

Open notepad and put this info in - note where things are comments and what variables you'll have to change:

Script start:

echo on
REM Set up some time variables
for /f "Tokens=1-4 Delims=/ " %%i in ('date /t') do set dt=%%l-%%k-%%j
for /f "Tokens=1" %%i in ('time /t') do set tm=-%%i
set tm=%tm::=-%
set dtt=%dt%%tm%
REM set up variables for log files, source and destination - change this variable
set log="C:\Users\owner\Documents\Scripts\Logs\%dt%.log"
REM local stuff to be backed up - change this variable
set src="c:\documents"
REM remote location to put backups - change this variable
set dest="I:\backups\server"
REM now for the actual work - change switches as required - explanation of switches is below.
robocopy %src% %dest% /E /Z /MIR /R:1 /LOG:%log%
REM I'd like to know how it went (this file can be big if there are a lot of files copied)
echo Backup Logs attached | blat - -subject "Sync Log Report for %dt%" -to "me@mydomain.com" -attach %log% -f user@domain.com

Use blat to send the email - grab it from www.blat.net (great program!) It sends an email with a header that will look like this:

Sync Log Report for 2012-02-10

and an attachment of your log file. You can add different things to this - for example I'll often use a [servername] tag after the date.

The robocopy switches used are:

• /E = copy sub-directories, including empty ones
• /Z = copy files in restartable mode (in case the network drops out or something similar)
• /MIR = MIRror a directory tree (which is /E plus /PURGE)
• /R:1 = number of retries on failed copies. It's best to set this - by default it's 1 million (!)

I run this from the Windows Scheduler and have a mirrored copy of data files each night. It's quite a useful little tool. If you'd like to use xcopy instead there are a few things to consider:

• the src and dest variables need to have a trailing backslash and a wildcard
• set src="c:\documents\*"
• set dest="i:\backups\server\*"
• and the command to insert would be:
• xcopy %src% %dest% /C /D /E /H /Y > %log%
• where the switches are:
• /C = continue copying even if there are errors
• /D = copies files whose source is newer
• /E = copies directories and sub-directories (even if empty)
• /H = copies hidden and system files
• /Y = suppresses prompting to overwrite files
• the > redirects xcopy's output to the %log% variable we configured earlier in the script, and then blat will email the resulting file out.

If you find this useful in anyway, please let me know in the comments.

Tips and Tricks: Monitoring with NTOP and IFTOP

For monitoring networks I have an old GX260 Dell desktop in the small form factor. Packed into this little device are a couple of network cards, added on are a wireless NIC and an extra USB network device. The purpose? To slot this in between the router and the network and see what's going backwards and forwards - very useful in the situation where a client is hemorrhaging bandwidth and doesn't know why. I'm running Ubuntu 10.04 LTS on it and it behaves very well.

Two of the main tools I use are ntop and iftop. For those of you not familiar with them, ntop monitors a particular interface and creates some nice webpages to be checked by the user in order to see what's going on through the network. iftop is similar but real time and is available through a console - which is the real appeal for me.

The server in question has two internal NICs, both are 100MB cards and are scripted to come up as a transparent bridge - br0. Basically I monitor that bridge and use one of the other interfaces to see what the server is seeing. Here is the script I use (adapted from elsewhere) to bring the bridge up:

#!/bin/bash
PATH="/sbin:/usr/sbin:/usr/local/sbin";
slaveIfs="1 2 3 4 6 7 8 9 10";
cmd="$1";
[ -z "$cmd" ] && cmd="start";
case "$cmd" in
  start)
    brctl addbr br0;
    brctl stp br0 on;
    brctl addif br0 eth1;
    brctl addif br0 eth2;
    (ifdown eth1 1>/dev/null 2>&1;);
    (ifdown eth2 1>/dev/null 2>&1;);
    ifconfig eth1 0.0.0.0 up;
    ifconfig eth2 0.0.0.0 up;
    ifconfig br0 up ### Adapt to your needs.
    ;;
  stop)
    brctl delif br0 eth1;
    brctl delif br0 eth2;
    ifconfig br0 down;
    brctl delbr br0;
    #ifup eth0; ### Adapt to your needs.
    #ifup eth1; ### Adapt to your needs.
    ;;
  restart,reload)
    $0 stop;
    sleep 3;
    $0 start;
    ;;
esac;

It resides in /etc/init.d and is called (imaginatively) bridge.sh and then has appropriate symlinks to rc2.d.

NTOP
NTOP is reasonably easy to configure on Ubuntu and is quite straightforward to get going - point it at http://localhost:3000 and set your username/password and off you go.

A tip to remember is that in the /etc/default/ntop it's a good idea to uncomment GETOPT="" and change it to read (if your network was 192.168.0.0/24):

GETOPT="--local-subnet=192.168.0.0/24"

and restart NTOP. Why is this important I hear you ask? Well on the br0 interface there is no IP assigned to it so NTOP doesn't automatically figure out what the local network is. By assigning this you can set it up to get proper info on local-remote, remote-remote and local-local traffic. Then leave it to run and see what it tells you.

IFTOP
Ah iftop is such a nice little bit of software. apt-get install it and then run it from the console with iftop -i br0 and it will tell you all sorts of things - data going from here to there and the level of bandwidth being used. I fancy pants it up a bit by creating a shell script and then launching it from there. iftop will use .iftoprc by default and if there isn't one will simply launch with it's own defaults. Here is the shell script I use (again I have adapted this from someone else):

#!/bin/sh

# customisable settings
LOCALNET="192.168.0.0/24"
IFACE="br0" # the bridged interface
CONF="/etc/iftoprc"

/usr/sbin/iftop -p -n -N -i $IFACE -F $LOCALNET -c $CONF

where /etc/iftoprc looks like this:

dns-resolution: yes
port-resolution: yes
show-bars: yes
promiscuous: no
port-display: source-only
#hide-source: yes
#hide-destination: yes
use-bytes: yes
sort: 2s
#line-display: one-line-both
show-totals: yes

Most of these are self-explanatory and I believe you should examine them more closely if you are looking to deploy it. Suffice to say, it gives me the info I want and I'm happy with that setup. So I hope that gives you some food for thought and you can take some of this away with you when you're trying to find out what the hell is chewing up all your bandwidth and download limit!