this is obsolete doc -- see http://doc.nethence.com/ instead

Poor man's daily backup tool 

Introduction 

Associated with a few scripts, basic UNIX tool may provide minimal but efficient backup techniques. Instead of TSM, TINA, Netbackup or Data Protector, one of those may be just enought: 

- LVM2 snapshots 

- dump/restore 

- tar/gtar 

- rsync 

- cpio 

Note. You need to exclude databases' datafiles from the backup tree for these. To backup databases, it's preferable to use their own backup tools: 

- RMAN for Oracle 

- mysqldump for MySQL 

Note. There is also complete free software solutions : 

- [Mondo Rescue] (http://www.mondorescue.org/) (Disaster recovery capable) 

- [Amanda] (http://www.amanda.org/) (Advanced Maryland Automatic Network Disk Archiver) 

- Bacula: http://www.bacula.org/en/ 

In this guide we are proceeding with the simplest solution ever, namely tarball backups executed by a daily cron job, plus a mirror executed from a remote host (see the "backing up the backups" section). 

Backup script example 

Create the backup directory, e.g., 

mkdir -p /data/backup/

and write a freaking simple backup script based on this example, 

#!/bin/ksh

#

# Poor man's backup tool

#

backupdir=/data/backup 

folders="/data/www /data/www.apache /home/MAILDIR_USER"

mysql_root_password=MYSQL_PASSWORD 

dateformat=`date +%Y-%m-%d-%H:%M:%S`

maxold=5

print cleaning up ${maxold}+ days old backups...\\c

find $backupdir/ -type f -mtime +${maxold}d \

        -name "*.gz" -o -name "*.bz2" | xargs rm -f && print done

print backup up all mysql databases...\\c

mysqldump -uroot -p"$mysql_root_password" --all-databases | gzip \

        > $backupdir/$dateformat.mysql.all.gz && print done

cd /

for folder in $folders; do

        relative=`echo $folder | sed 's|^/||'`

        folder_=`echo $folder | sed 's|^/||; s|/|_|g'`

        print backing up $folder/...\\c

        tar -czp --exclude "lab.nethence.com/mirrors/*" \

                -f $backupdir/$dateformat.$folder_.tar.gz $relative && print done

        unset relative folder_

done; unset folder

print ''

ls -alhF $backupdir/

print ''

/root/bin/backup_lftp.ksh

Enabling the cron job 

Edit root's crontab, 

crontab -e

like e.g., 

SHELL=/bin/ksh

HOME=/root

PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:/root/bin

LANG=en_US.UTF-8

MAILTO=root

# Note. the /root variable isn't defined inside cron by default, cannot use

# HOME=$HOME.  Cannot even use $HOME within the PATH variable.

0 4 * * * backup.ksh

Note. On FreeBSD the crontabs which get edited as /etc/crontab instead of running the crontab executable, are multi-user capable (with the additional 6th field). So you shouldn't add /root/bin in the path, and you should use absolute path for user's own executables ($HOME is allowed there and only there -- not in the PATH). 

Note. "0" and "7" both correspond to Sunday. 

Archiving the daily backups remotely 

I prefer to launch the archiving jobs from a host that is dedicated to that, preferably with lots of disk space, so if the target server get's compromised, its backups won't. You can proceed with [rsync](http://pbraun.nethence.com/unix/net/rsync.html) or FTP, using a server on each nodes e.g. [tnftpd] (http://pbraun.nethence.com/unix/net/ftpd.html) and a client script on the dedicated host for the backup archives. 

Setting up a reverse FTP mirror with Lftp 

However if you only have access to an FTP account to archive your backups, you don't have any other choice than to use a client script to connect to it. You can use this example script, 

#!/bin/ksh

        print checking for previous backup processes...\\c

        [[ -n `pgrep lftp` ]] && print PREVIOUS FTP REVERSE MIRROR IS STILL RUNNING \

                && print "$tmp" && exit 1

        print done

        print calculating needed space...\\c

        needed=`du -s /data/backup/ | awk '{print $1}'`

        print $needed

avail=10485760 

        print available space is $avail

        (($avail <= $needed)) && print NO SPACE LEFT ON REMOTE SITE && exit 1

        lftp -f "

open FTP_SERVER 

user FTP_USER FTP_PASS 

lcd /data/backup

mirror -R -c --delete-first --parallel=2

bye

"

#date=`date +%Y-%m-%d-`

#lftp: rm -rf /

#lftp: mput -c ${date}*

and call it at the end of your backup script. 

Other references 

Backup & Recovery: http://oreilly.com/catalog/9780596102463/ 

NetBSD dump manual: http://netbsd.gw.com/cgi-bin/man-cgi?dump++NetBSD-current 

NetBSD restore manual: http://netbsd.gw.com/cgi-bin/man-cgi?restore++NetBSD-current