Nethence Newdoc Olddoc Lab Your IP BBDock  


Warning: those guides are mostly obsolete, please have a look at the new documentation.

UnixWindowsOracleObsoleteHardwareDIYMechanicsScriptsConfigs

Poor man's daily backup tool
 
Introduction
Associated with a few scripts, basic UNIX tool may provide minimal but efficient backup techniques. Instead of TSM, TINA, Netbackup or Data Protector, one of those may be just enought:
- LVM2 snapshots
- dump/restore
- tar/gtar
- rsync
- cpio
Note. You need to exclude databases' datafiles from the backup tree for these. To backup databases, it's preferable to use their own backup tools:
- RMAN for Oracle
- mysqldump for MySQL
 
Note. There is also complete free software solutions :
- Mondo Rescue (Disaster recovery capable)
- Amanda (Advanced Maryland Automatic Network Disk Archiver)
- Bacula: http://www.bacula.org/en/
 
In this guide we are proceeding with the simplest solution ever, namely tarball backups executed by a daily cron job, plus a mirror executed from a remote host (see the "backing up the backups" section).
 
Backup script example
Create the backup directory, e.g.,
mkdir -p /data/backup/
and write a freaking simple backup script based on this example,
#!/bin/ksh
#
# Poor man's backup tool
#
backupdir=/data/backup 
folders="/data/www /data/www.apache /home/MAILDIR_USER"
mysql_root_password=MYSQL_PASSWORD 
dateformat=`date +%Y-%m-%d-%H:%M:%S`
maxold=5
 
print cleaning up ${maxold}+ days old backups...\\c
find $backupdir/ -type f -mtime +${maxold}d \
-name "*.gz" -o -name "*.bz2" | xargs rm -f && print done
 
print backup up all mysql databases...\\c
mysqldump -uroot -p"$mysql_root_password" --all-databases | gzip \
> $backupdir/$dateformat.mysql.all.gz && print done
 
cd /
for folder in $folders; do
relative=`echo $folder | sed 's|^/||'`
folder_=`echo $folder | sed 's|^/||; s|/|_|g'`
 
print backing up $folder/...\\c
tar -czp --exclude "lab.nethence.com/mirrors/*" \
-f $backupdir/$dateformat.$folder_.tar.gz $relative && print done
unset relative folder_
done; unset folder
print ''
 
ls -alhF $backupdir/
print ''
 
/root/bin/backup_lftp.ksh
 
Enabling the cron job
Edit root's crontab,
crontab -e
like e.g.,
SHELL=/bin/ksh
HOME=/root
PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:/root/bin
LANG=en_US.UTF-8
MAILTO=root
# Note. the /root variable isn't defined inside cron by default, cannot use
# HOME=$HOME. Cannot even use $HOME within the PATH variable.
0 4 * * * backup.ksh
Note. On FreeBSD the crontabs which get edited as /etc/crontab instead of running the crontab executable, are multi-user capable (with the additional 6th field). So you shouldn't add /root/bin in the path, and you should use absolute path for user's own executables ($HOME is allowed there and only there -- not in the PATH).
Note. "0" and "7" both correspond to Sunday.
 
Archiving the daily backups remotely
I prefer to launch the archiving jobs from a host that is dedicated to that, preferably with lots of disk space, so if the target server get's compromised, its backups won't. You can proceed with rsync or FTP, using a server on each nodes e.g. tnftpd and a client script on the dedicated host for the backup archives.
 
Setting up a reverse FTP mirror with Lftp
However if you only have access to an FTP account to archive your backups, you don't have any other choice than to use a client script to connect to it. You can use this example script,
#!/bin/ksh
 
print checking for previous backup processes...\\c
[[ -n `pgrep lftp` ]] && print PREVIOUS FTP REVERSE MIRROR IS STILL RUNNING \
&& print "$tmp" && exit 1
print done
 
print calculating needed space...\\c
needed=`du -s /data/backup/ | awk '{print $1}'`
print $needed
 
avail=10485760 
print available space is $avail
 
(($avail <= $needed)) && print NO SPACE LEFT ON REMOTE SITE && exit 1
 
lftp -f "
open FTP_SERVER 
user FTP_USER FTP_PASS 
lcd /data/backup
mirror -R -c --delete-first --parallel=2
bye
"
#date=`date +%Y-%m-%d-`
#lftp: rm -rf /
#lftp: mput -c ${date}*
and call it at the end of your backup script.
 
Other references
Backup & Recovery: http://oreilly.com/catalog/9780596102463/
NetBSD dump manual: http://netbsd.gw.com/cgi-bin/man-cgi?dump++NetBSD-current
NetBSD restore manual: http://netbsd.gw.com/cgi-bin/man-cgi?restore++NetBSD-current
 

(obsolete, see the new doc)