Until recently the extent of my backup efforts was to take the occasional CD copy of my home directory and keep copies of important files somewhere else, usually on another disk partition or a floppy disk. All of this changed with the need to run some Windows legacy applications. The only machine really suitable for this work was my main workstation, a 1.2GHz Athlon machine, multibooted with four distributions. I decided to free up the first primary partition, which held Mandrake 9.0, and set up a Windows partition.

I freed up the first primary partition by transferring its contents to the seventh partition, overwriting an expendable Vector Linux 3.0 Distribution. To be totally safe, I booted into Debian 3.0 and mounted both partitions to individual mount points in /mnt. Then, as root, I used tar and a pipe to copy everything, including all links and permissions, from the source partition to the target partition. A few minutes later, after changing my GRUB boot menu, I was able to boot into Mandrake 9.0 Linux in the seventh partition and verify that everything worked as expected.

At this point, one normally would DOS-format the now free first partition and install Windows. I began to feel a little uneasy, however. Windows could format the whole darn drive or some other similar screwup could happen, in which case I would be in the position of having to fdisk the partitions and reinstall everything from scratch. The original disks, of course, would have all the applications except for those extra packages installed by me, but all custom configurations would be lost.

The machine now was running Mandrake 9.0, Debian 3.0 and Slackware 8.1. Of these, only losing my Slackware install would cause me grief, as it has been running like a top, boots to KDE 3.0 in less than 30 seconds--including my sign on--and is absolutely rock-solid stable. It also has the CUPS printing system set up perfectly for all of my printers on the LAN. So I must retain this setup at all costs. The solution, of course, is to back up fully everything from the Slackware install.

At this point, the desire to have a simple, easy and foolproof backup and recovery method took hold.

If you are a home or SOHO Linux user, I suggest that your backup and recovery system should:

A quick review of past Linux Gazette articles and a search of the Web turn up hundreds of backup solutions. Many are aimed specifically at the backup function; others are aimed at the repair and system recovery part of the overall effort to get back to some predefined state. Virtually none are customized for your system or your specific requirements, so why not roll your own solution? That is what we do here.

Most home or SOHO users do not have a tape drive system and are unlikely to purchase one for the sole purpose of backup, given that the cost of the tape system and software probably exceeds that of the computer itself. This essentially narrows our options to backup to removable disk, backup to the same or another hard drive, backup to a CD or backup over a network to some other hard drive. This last option essentially is a more complicated backup to local hard drive option, except there is zero chance of it being lost when your system goes down. So let us look at these options:

The most essential requirement is to have a working and reliable CD burner. Any current Linux distribution has the required tools, and to minimize media costs, about $4 supplies two good quality CD-RW disks. For daily backups, these will last for about five and a half years, and used weekly, they will last for eternity.

The scheme proposed here is to use the two CD-RW disks to take backups in rotation. In my actual implementation, I have color-coded the spine of the disk covers red and green, respectively, to aid in the correct rotation.

We also require the backup disk to self-boot into a minimal working Linux system. This is necessary to ensure that we can re-establish the master boot record (MBR) and the rest of the original partition information, if required. This rules out using a boot disk image, as commonly supplied with the majority of distributions. These supply only a boot method and a Linux kernel, and they usually boot straight to the partition they are customized to boot.

After a quick perusal of the Linux on self-boot CDs available, I settled on using the classic and well tried TomsRtBt disk in a 2.88MB image format. This is not an ISO image, but it is suitable for being the boot image of an ISO we will burn. It is also to be found at various other sources on the Web. I have used this self-boot in the floppy format, and it is good and quite complete. Note that it also includes a Toms FAQ.

In order to restore our working Linux system to a given state, we need to have records of all the current directory contents that are changing on a day-to-day basis or have changed as customizations since the initial install. This can be done laboriously by inspection and detailed lists, which minimizes what must be restored. Or, it can be accomplished easily by backing up the entire contents of these directories.

In my case, I have decided to back up the entire contents of */home, /etc, /usr/local, /opt, /var, /root and /boot* on the Slackware 8.1 partition:

In addition to the contents of each of the identified directories above, there are some more important pieces of information that one wouldn't want to be without if a sudden failure to boot occurred. These include a binary copy of the MBR, a text list of the partition table and a copy of the fstab file, in case you have forgotten which partitions correspond to which filesystem. Optionally, you could make a copy of the current XF86Config file and/or the text output of commands such as lsdev and lspci for full system information.

Also, how are we going to structure all of this information to ensure it gets onto the CD in such a way as to be completely self-contained and usable for the task at hand? Here is what I did. First, I created a directory to hold all of the information to back up. As root, I issued /mkdir /tmp/backup/. Note here that I am using /tmp as repository for the constant part of the backup CD. This is safe in Slackware, but it might not be in other distributions. Choose a safe location, and one not itself backed up by the tar file.

Put into the backup directory a copy of the TomsRtBt .img file

cp ./tomsrtbt288.img /tmp/backup/tomsrtbt288.img

Here the .img file is in my home directory.

Put into the backup directory a copy of the master boot record:

dd if=/dev/hda bs=512 count=1 > /tmp/backup/MBR.bin

The MBR holds the first stage of the boot mechanism you employ. In my case, that's stage1 of GRUB, the GRand Unified Boot Loader, or LILO, s well as the partition information for the primary partitions. The extended partition information is held elsewhere on the disk and can be restored--if required--with the information you store from the fdisk command detailed next.

Put into the backup directory a list of the partition information:

fdisk -l > /tmp/backup/Partition_Table

This will be used to compare with a Toms listing of the partition table before any restoration takes place.

Put into the backup directory a copy of fstab, which defines the file system mount points. Be aware that any errors here will make the files and devices inaccessible later:

cp /etc/fstab /tmp/backup/fstab.bak

Optionally, copy any other information you wish to have available to you before you are able to boot into your newly restored Linux system. For easy accessibility, I keep a copy of XF86Config on the disk to ensure that I always can set up X the way I like, even if I'm installing a new system upgrade. I also keep a copy of menu.lst, as I use GRUB as my boot loader of choice:

cp /etc/X11/XF86Config /tmp/backup/XF86Config.bak 
... 
cp /boot/grub/menu.lst /tmp/backup/menu.lst.bak

These files are added to every copy of the backup disk that is burned. The disks need to be changed only if one of the files changes, and then it should be copied over.

Now we need to ask: what do we need to do to create our self-booting backup disk? Here's the procedure I used:

The script to accomplish all of these steps is shown below. Be sure to make it executable--chmod 755 ./backup--and copy it to somewhere in root's PATH; /usr/local/bin is a good place. Do the same for the next script as well.

#!/bin/bash
#  backup
#------------------------------------------------------------------------------
#  Script to enable easy backup of all important Linux files
#  and also creates a customized system repair disk.
#  Uses two CD-RW Disks labled "RED" and "GREEN to rotate backups
#------------------------------------------------------------------------------
# The backup directory already contains files for boot and recovery.
# One can add more - my Slackware 8.1 system backup is < 580MB.

Backup_Dirs="/home /etc /usr/local /opt /var /root /boot"
Backup_Dest_Dir=/tmp/backup
Backup_Date=`date +%b%d%Y`
Image_File=/tmp/backup.iso
declare -i Size

# Create tar file with todays Month Day Year prepended for easy identification
tar cvzf $Backup_Dest_Dir/$Backup_Date.tar.gz $Backup_Dirs &> /dev/null

# Start backup process to local CD-RW drive
echo "Backing up $Backup_Dest_Dir to CD-RW Drive - $Backup_Date"
echo "Creating ISO9660 file system image ($Image_File)."

mkisofs -b toms288.img -c boot.cat -r \
        -o $Image_File $Backup_Dest_Dir  &> /dev/nul

# Check size of directory to burn in MB
Size=`du -m $Image_File | cut -c 1-3`
if [ $Size -lt 650 ]
then
   echo "Size of ISO Image $Size MB, OK to Burn"
else
   echo "Size of ISO Backup Image too Large to burn"
   rm $Backup_Dest_Dir/$Backup_Date.tar.gz # Remove dated tar file
   rm $Image_File   # ISO is overwritten next backup but cleanup anyway
   exit 1
fi

# Burn the CD-RW
Speed=4                 # Use best speed for CD-RW disks on YOUR system
echo "Burning the disk."
                              # Set dev=x,x,x from cdrecord -scanbus
cdrecord -v speed=$Speed blank=fast dev=1,0,0 $Image_File &> /dev/null
Md5sum_Iso=`md5sum $Image_File`
echo "The md5sum of the created ISO is $Md5sum_Iso"

# Could TEST here using Md5sum_Iso to verify md5sums but problem is tricky.
echo "To verify use script md5scd, this will produce the burned CD's md5sum"
echo "run this as User with backup CD in drive to be used for recovery."
echo "This verifies not only the md5sum but that disk will read OK when needed."

# Remove image file and tar file
echo "Removing $Image_File"
rm $Image_File
echo "Removing : $Backup_Dest_Dir/$Backup_Date.tar.gz"
rm $Backup_Dest_Dir/$Backup_Date.tar.gz
echo "END BACKUP $Backup_Date"
echo "Be sure to place this backup in the RED CD case and previous CD in GREEN"
echo "------------------------------------------------------------------------"
exit 0

Once established, the backup process is simple. I usually back up every day. If I'm not doing much on the system, then I back up every week instead. At the start of every backup, I place the CD-ROM from the green CD case into the CD burner. In an xterm I su to root and issue this command:

nohup backup &> /tmp/backup.log &/

and then close the xterm and go to bed. The backup takes about 15 minutes to complete, so it can be done at any convenient time in a work day. When next at the computer, I cat /tmp/backup.log and check that the backup went well.

If I also want to verify the backup ISO, I note down the first and last four or five letters of the listed ISO md5sum. As my burner does not reliably read back the CD just written, I transfer the CD to my CD-ROM drive and verify that the md5sums are identical, using the script md5scd (see the listing below). If they are identical, I put that newly burned CD into the red case and the last burned CD into the green case, ready for the next backup cycle. If any possibility of confusion exists, one can always check the date of the tar file. Note that because of the burner not reliably reading the backup CD, I have not included an automatic check of the md5sums. I did this because failure to validate does not mean the CD is in error, only what was just read from the burner was in error. In fact, I have never experienced a md5sum compare failure when using my CD-ROM. I consider the MD5 checksum to be essential, as even a single-bit error conceivably could corrupt the entire compressed archive.

#!/bin/bash
#------------------------------------------------------------------------
# md5scd ---- Data CD md5sum Verifier
# Script to automate determining Md5sum for ISO9660 CDs
# NOTE - This script assumes that correct md5sum is known and one wishes
# to verify that a particular CD copy has been burnt correctly.
# If working from a downloaded ISO image use "md5sum ISO" at command line
#------------------------------------------------------------------------
# Requires standard tools found in all Linux Distributions
# If script invoked as user, check all permissions, groups, etc.

# Missing arguments?
if [ $# -ne 2 ]
then
  echo "Usage - md5scd mountpoint device, ex - md5scd /mnt/cdrom /dev/hdc"
  exit 1
else
    : OK have arguments
fi

# Loop over md5sum determination ...100 good copies for install-fest?
again=yes
while [ "$again" = yes ]
   do
     echo "Please insert CD at $1 and press ENTER when ready"
     read                        #Wait for insertion of disk
     mount $1
     Block_Count=`df -k $1 | grep $1 | cut -c 25-30`
               #700Mb disk cannot exceed this ^^^^^ column limit in 1k blocks
     umount $1
     Md5sum_Cd=`dd if=$2 count=$Block_Count bs=1024 | md5sum`
     echo "The md5sum of the CD at $1 is " $Md5sum_Cd
     echo
     echo -n "Verify another CD? [yes/no]"
     read again   # Wait for "yes" -> repeat, anything else -> drop through'
   done
exit 0

Before that eventuality occurs, one should make sure the backup disk will boot. Also, make sure you understand the limitations of tomsrtbt, and practice reading the various files placed on the backup disk, as the only editor available there is Vi. When you need it, the disk first has to be mounted:

mount -t iso9660 /dev/xxx /mnt

It is possible to unzip and untar the tarred file using TomsRtBt by first using gzip and then tar. However, it probably is better to first check that the partition table is correct by using fdisk -l and a reference to the stored table. If not, to restore the MBR use:

dd if=/mnt/MBR.bin of=/dev/hda bs=1 count=512 

This restores the primary partitions and the bootloader. Then, use fdisk and the partition table file to restore the extended partition manually along with the logical partitions within it. This all requires nerve and practice! However, any changes can be abandoned if you're not sure or are only practicing.

Next, do a clean install to the proper partitions. Reboot to the point where one has a root console, mount the backup CD and execute

tar xvzf /Mount_Dir/Backup_Tar_Filename

This restores all of the previous directories to their correct places. It should leave you with an almost fully restored system.

Note that if the problem is only lost or corrupt files, one also can restore any of the saved directories at any time by extracting with, for example:

tar xvzf /Mount_Dir/Backup_Tar_Filename /home

The test of any system is, "Does it work?" I initially verified that the backup CD does boot into Toms wonderful Linux system, that all of the text material was readable and that fdisk -l did correspond to the stored version. I did not reinstate the MBR from the binary image file, but it is there if I ever need it.

The final test was to restore my Slackware 8.1 system in place of the original Mandrake 9.0 system, before installing Windows and perhaps having to restore. In brief:

In another five minutes, this reinstalled all of the backed up partition contents. A reboot now brought me into a restored Slackware 8.1 with X set up and a KDE login. Because of not saving /dev, some of the device permissions had to be reset--sound and so on--but this was trivial. The whole process took less than half an hour, a far cry from doing a normal install and adding all of the missing favourite programs, followed by a lengthy reconfiguration.

Backup is easy to do and easy to keep doing. Now that the system is in use, I see that a number of small improvements could be made. The manual backup and verification commands could be made shell variables and invoked with a single word. Also, if the total file size becomes a factor, one could use the --exclude option of tar to not include large sections of invariant code in the tar file. Alternatively, one could use bzip2 compression. As it is now, complete directories from root are saved.

The urgent need for the Windows applications turned out to be not so urgent, but the occasion gave me the prod I needed to actually back up regularly. Perhaps my next project will be to install Wine and try to get those pesky applications to run within Linux, without the need to always reboot.

I would be interested in any comments for improvement. Indeed any feedback would be welcome, particularly if glaring flaws or omissions are evident. This scheme has been in use for only a short time, but so far I'm very satisfied. Finally, I encourage you also to institute regular backups. If you want a quick ready-made approach, try this one. A few changes to the scripts should have you backing up today and everyday after that.

Copyright (c) 2003, Alan Keates. Originally published in Linux Gazette issue 91. Copyright (c) 2003, Specialized Systems Consultants, Inc.