UNIX / How to configure an FTP mirror

Home | UNIX | Practical

How to configure an FTP mirror


Introduction
Mirror is a perl 5 program to replicate FTP repositories.
Source tarball : src.doc.ic.ac.uk/computing/archiving/mirror/mirror-2.9.tar.gz
NetBSD package : pkgsrc.se/net/mirror


Configuration
Edit the configuration,
vi path/lib/mirror.defaults
For example,
package=defaults
hostname=ftp.nethence.com
local_dir=/PATH/TO/FTP/mirrors/WTF
local_dir_check=false
remote_password=YOUREMAIL@example.com
mail_to=pbraun@nethence.com
dir_mode=0755
file_mode=0444
user=1000
group=1003
update_log=.mirror
#exclude_patt=(^|/)(\.mirror$|\.mirror\.log|core$|\.cap|\.in\..*\.$|MIRROR\.LOG|#.*#|\.FSP|\.cache|\.zipped|\.notar|\.message|lost\+found/|Network Trash Folder)|suky.mpe?g
exclude_patt=\.mirror$|\.cache|\.message|lost\+found/
compress_patt=
compress_prog=gzip
compress_excl+|-z(\d+)?$|\.tgz|_tgz|\.tar\.Z|\.tar\.gz|\.taz$|\.arc$|\.zip$|\.lzh$|\.zoo$|\.exe$|\.lha$|\.zom$|\.gif$|\.jpeg$|\.jpg$|\.mpeg$|\.au$|\.shk$|rpm$|deb$|read.*me|index|info|faq|gzip|compress|(^|/)\.\.?$
delete_excl=(^|/)\.(mirror|notar|cache)$
local_ignore=(^|/)\.(mirror|notar)$
do_deletes=true
max_delete_files=900
max_delete_dirs=900
timeout=3000
failed_gets_excl=\:\ Permission denied\.$
passive_ftp=1
algorithm=1

To create a new FTP mirror, e.g. from linuxpackages.net,
vi path/lib/linuxpackages.net
For example,
package=linuxpackages.net
comment=linuxpackages.net
site=ftp.scarlet.be
remote_dir=/pub/linuxpackages/Slackware-11.0
local_dir=/PATH/TO/FTP/mirrors/linuxpackages.net/Slackware-11.0
exclude_patt=\.mirror$|\.cache|\.message|lost\+found/


Usage
Then launch mirror with specific site's config as argument,
mirror path/lib/linuxpackages.net



Sat Nov 8 12:19:48 CET 2008
       © 2008 Pierre-Philipp Braun