Nethence Newdoc Olddoc Lab Your IP BBDock  


Warning: those guides are mostly obsolete, please have a look at the new documentation.

UnixWindowsOracleObsoleteHardwareDIYMechanicsScriptsConfigs

Configuring the procmail Mail Delivery Agent
 
http://pbraun.nethence.com/unix/mail/procmail.html
http://pbraun.nethence.com/unix/mail/procmail-bmf.html
http://pbraun.nethence.com/unix/mail/procmail-qsf.html
 
Introduction (update)
It is a good idea to use the Maildir format instead of traditional mbox. It allows several imap connections to read and write to the folders at the same time. Also, it's faster than mbox as each email is stored in a seperate file (mbox need to read the whole file to show the messages in a folder...). Finally, it still creates folders that does not exist, even sub-sub folders (subfolder are in the .sub.folder layout anyway), just like with traditionnal procmail rules.
 
To use Maildir, use e.g. this as destination (note the dot prefix and the trailing slash)
.folder/
and for subfolders,
.folder.sub/
and for the INBOX and default folder,
ORGMAIL=$HOME/Maildir/
MAILDIR=$HOME/Maildir
DEFAULT=$ORGMAIL
 
Requirements
Make sure procmail is installed on your system,
which procmail
 
System configuration
There are basically three ways for using procmail :
 
- As an SMTP daemon MDA
If you're getting messages from your smtp daemon,
vi ~/.forward
like,
"|IFS=' '&&p=/usr/local/bin/procmail&&test -f $p&&exec $p -f-||exit 75"
Ref. http://mirror.ncsa.illinois.edu/procmail-faq/mini-faq.html#forward
 
- As a POP receiver MDA
If you're getting message for several POP accounts, it's preferable to make a clean cut separation between the procmail configs and call procmail with -m (with fetchmail or getmail).
 
- Manually or by some script
For example, see the "Reprocessing messages" chapter below.
 
procmail rules examples
Here are some procmail configuration examples,
cd ~/
wget http://pbraun.nethence.com/configs/mail/.procmailrc
wget http://pbraun.nethence.com/configs/mail/.procmailrc.trash
 
Refs.
11.6 Matching last Received header: http://www.hcs.harvard.edu/~thurston/ua/pm-tips.html#matching_last_received_header
 
Note. yes, SHELL=/bin/ksh permits to use the KSH syntax e.g.,
* ? [[ -f lists/file1 || -f lists/file2 ]]
and
* !? [[ -f lists/file1 || -f lists/file2 ]]
 
Note. DO NOT use the signature dashes wiper, it breaks attached PDF documents,
# DO NOT USE : THIS BRAKES PDF ATTACHMENTS
# Fix signature dashes
#:0 fBw
#* ^--$
#| sed -e 's/^--$/-- /'
 
Note. To backup incoming messages by date,
yearmonth=`date +%Y%m`
:0 c
backup.$yearmonth
 
Using Maildir format
Note. no colon after :0 on Maildir folders, and the trailing slash (/) is mandatory.
 
Note. Maildir folder based on dots is better than filesystem based because you do not have to care about the folder existing or not, when writing a procmail rule,
:0
* ^From:.*(Charlie Root|Cron Daemon|logwatch@)
.archives.$YEAR.crontabs/
will just create the folder in case it doesn't exist (which is a file, that is the point).
 
Rewrite subject with formail
One-liner to change the subject header,
:0 fh
* Subject: \/.*
| formail -I "Subject: [some text] $MATCH"
 
But if you already need the MATCH variable for something, you can also use a seperate rule to isolete a 'subject' variable,
:0 chfw
subject=| /usr/bin/formail -xSubject:
 
:0 Hf
| /usr/bin/formail -i "Subject: [$MATCH] $subject"
Note. 'chfw' to forge a variable out of some header
Note. 'Hf' to pipe the message to formail
 
Clean-cut POP accounts separation
You may use condition checks against the Received header,
:0
* ^Received:.*IP_OR_HOSTNAME
{
Note. for that, enable your POP fetcher's ability to add its own Received header
 
Misc syntax tips
Note. exclude some messages e.g. messages having a spam-like Reply-To,
* !^Reply-To:.*@spamdomaine.com
Note. ':0 Wic' for piping to mkdir (man procmailex)
Note. ':0E' for procmail 'else if' condition
 
Other procmail configuration examples:
http://spacepants.org/conf/dot.procmailrc
http://puzzling.org/computing/help/procmail
http://bethesignal.org/dotfiles/procmailrc.html
 
Reprocessing messages (mbox and Maildir)
Installation
As the username which has the .procmailrc file and which will use it, add ~/bin to its path and fetch the reprocess script,
cd ~/
mkdir -p bin/
cd bin/
wget http://www.uq.edu.au/~suter/software/reprocess-mbox-via-procmail/reprocess-mbox-via-procmail
chmod +x reprocess-mbox-via-procmail
Note. formaly found on http://zwitterion.org/software/reprocess-mbox-via-procmail/.
 
Also provide a wrapper for Maildir folders,
vi reprocess_maildir
like,
#!/bin/ksh
#
# Re-process a Maildir format mailbox with Procmail
#
set -e
 
[[ -z $1 ]] && print \$1 missing && exit 1
[[ ! -d $1 ]] && print folder $1 does not exist && exit 1
[[ ! -r $1 ]] && print folder $1 is not readable && exit 1
 
find $1/cur $1/new -type f | while read msg; do
print Processing $msg...\\c
procmail < $msg && rm -f $msg && print Done
done
 
Configuration
Eventually adapt the called procmailrc file accordingly,
vi reprocess-mbox-via-procmail
change,
formail -s procmail -m /PATH/TO/.procmailrc < $tmpfile || die $0: formail failed - data in $tmpfile.
 
On NetBSD, you'll also have to make sure those packages are installed (not the base system mktemp, we need the sysutils/mktemp one),
pkg_info | grep bash
pkg_info | grep mktemp
and fix the script,
cd ~/bin/
vi reprocess-mbox-via-procmail
change '#!/bin/bash' and add /usr/pkg/{bin,sbin} to the PATH declaration.
 
Usage
As the username which has access to the .procmailrc, reprocess an mbox like,
cd /var/spool/virtual/example.net/user.imap/
reprocess-mbox-via-procmail MBOX_FILENAME
 
Alternatives
Use 'formail -s' to split the messages,
formail -ds procmail -Y -m $HOME/.procmailrc < mbox.input
Note. eventually use '-d' to handle non-standard message formats
Note. -d and -Y to deal with traditionnal mail format. Otherwise you may get thousands of times the same email, or just one email for the whole mbox.
 
Or simply resend the messages through sendmail/procmail,
cat mbox | usr/sbin/sendmail -oi user@example.com
 
Removing duplicates from an mbox
Get the perl script from Bill's webpage hacks to remove duplicates. Be careful, it's in DOS format! Convert with e.g. dos2unix.
mkdir -p ~/bin/
cd ~/bin/
#wget http://wdr1.com/projects/mbox-dedup.pl
wget http://pbraun.nethence.com/scripts/mail/mbox-dedup.pl
dos2unix mbox-dedup.pl
chmod +x mbox-dedup.pl
Note. eventually fix the perl path (using NetBSD?...),
#vi mbox-dedup.pl
Note. Many thanks to William Reardon for this script.
 
Remove duplicates,
cd /var/spool/virtual/example.net/user.imap/
#mbox-dedup.pl --help
mbox-dedup.pl mbox
rm -f mbox.bak
 
Otherwise, directly within the procmail rules,
:0 Wh: msgid.lock
| /usr/bin/formail -D 8192 /var/log/mailuser/msgid-noduplicate.cache
 
 
Troubbleshooting
Note. I didn't had much success with that one,
:0 Wh: msgid.lock
| /usr/bin/formail -D 8192 /var/log/mailuser/msgid.cache
 
References
http://userpages.umbc.edu/~ian/procmail.html
http://partmaps.org/era/procmail/mini-faq.html
http://lipas.uwasa.fi/~ts/info/proctips.html#orrules
http://pm-doc.sourceforge.net/doc/
http://sial.org/howto/procmail/
Great guide : http://partmaps.org/era/procmail/quickref.html
Sweet regex manual : http://www.perlcode.org/tutorials/procmail/regular_expressions.pod
Some mail headers : http://people.dsv.su.se/~jpalme/ietf/mail-headers/mail-headers.txt
Gentoo procmail : http://gentoo-wiki.com/Procmail_configuration
Timo's procmail tips and recipes : http://lipas.uwasa.fi/~ts/info/proctips.html
http://lists.debian.org/debian-user/2001/12/msg02530.html
http://luv.asn.au/overheads/procmail.html
http://tldp.org/LDP/LG/issue14/procmail.html
 

(obsolete, see the new doc)