Setting up a local IMAP archive of your mail on Tiger

The following instructions will allow you to create a local (ie. stored on your own hard disk) archive of email, that you can share between multiple email clients. Use at your own risk.

The Problem

Mail.app (certainly on OS X 10.4 onwards) does not read maildir or mbox format mail archives, at least for everyday use. You can "import" mbox formats, but once you do, they can no longer be accessed by mutt, thunderbird, or other mail clients. The following instructions will allow you to keep an archive of email on your local machine that can be accessed by any mail client, allowing you to choose whatever tool is best for the task in hand.

I'm going to assume that you are using mbox folders. I know there are many good reasons not to, but since I'm still living in the mbox timewarp. The following should work with minor modifications for maildir.

If anyone gets maildir working, let me know what needs changing.

Notes

We are going to set up an imap server, but just to serve mail to localhost.

Security will be provided by the firewall.

By default, if using Apple's firewall, you do not need to allow access to either of the IMAP ports - 143 for standard IMAP4, and 993 for SSL. This is because the default firewall allows access to any ports on localhost.

However, you should certainly make sure that you have a working firewall before following the following instructions. An IMAP server is essentially a file-browser, and you will be logging in with your standard user (though you could set up a separate user, if you wanted). Without the firewall in place, a security flaw in uw-imap would give an attacker access to your user files, and possibly to your whole system.

With a firewall in place, blocking remote access to these ports, you can be confident about using imap locally.

If you are sure that you will only ever be using imap over the loopback interface, you probably can use the slightly simpler non-ssl method. However, if you might ever want to the server remotely, follow the instructions for SSL.

Installing

Download the latest version of uw-imap from The University of Washington. At time of writing, this is version imap-2004g. Unpack that folder in a convenient place, such as your Desktop.

From the terminal: cd Desktop/imap-2004g

Compiling is simple.

If you want to use non-ssl IMAP and SSL IMAP, compile using the command:

make oxp PASSWDTYPE=pam SSLTYPE=unix

However, if you will be using SSL (recommended) just type type:

make oxp

The "oxp" tells uw-imap that you are compiling for OS X.

When that is done, copy the executable somewhere useful, with:

sudo cp imapd/imapd /usr/local/libexec/imapd

Now you need to set up PAM to be aware of imap. Mercifully, this is simple. You can simply copy the pam login file for imap:

sudo cp /etc/pam.d/login /etc/pam.d/imap

Now, you need to set up OS X to respond to connections on the relevant ports. With Tiger, this is done with launchd. See Apple's documentation for more details. An excellent editor for lauchd configuration is Lingon, which at time of writing is open source and version 1.0.

If you do use that interface, we will be creating a "User Daemon". This is apple's term for services installed by system administrators, as opposed to "System Daemons" that are Apple's own. You could create a System Daemon, but there is no need and it is unwise.

An advantage of using Lingon is that it provides an easy way to get launchd to re-read the configuration files for a particular service.

If you want to create the launchd files by hand, they live in /Library/LaunchDaemons/. By convention, you should name them with reverse domain-name notation. In this case, for example, edu.washington.imap or edu.washington.imaps. If you want to enable both SSL and non-SSL connections, you should create two services, otherwise just one of the following is needed. Either create the following files with a text editor, or cut and past this text into the "Expert" Tab of a new User Daemon in Lingon. This is the recommended method.

For non-ssl (port 443) IMAP:

<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>Label</key> <string>edu.washington.imapd</string> <key>OnDemand</key> <true/> <key>ProgramArguments</key> <array> <string>/usr/local/libexec/imapd</string> </array> <key>Sockets</key> <dict> <key>Listeners</key> <dict> <key>SockServiceName</key> <string>imap</string> <key>SockType</key> <string>stream</string> </dict> </dict> <key>inetdCompatibility</key> <dict> <key>Wait</key> <false/> </dict> </dict> </plist>

For SSL IMAP

<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>Label</key> <string>edu.washington.imaps</string> <key>OnDemand</key> <true/> <key>ProgramArguments</key> <array> <string>/usr/local/libexec/imapd</string> </array> <key>Sockets</key> <dict> <key>Listeners</key> <dict> <key>SockServiceName</key> <string>imaps</string> <key>SockType</key> <string>stream</string> </dict> </dict> <key>inetdCompatibility</key> <dict> <key>Wait</key> <false/> </dict> </dict> </plist>

Once that is down, unload and load the service with Lingon (reload doesn't always seem to work), or restart launchd as follows.

To reload the config files from a terminal (with thanks to AJJ for pointing out launchctl), do:


sudo launchctl unload -w /Library/LaunchDaemons/edu.washington.imap
sudo launchctl load -w /Library/LaunchDaemons/edu.washington.imap

You don't need the first step if loading the config for the first time, of course, and you should use whatever filenames you have given the services.

If you are using SSL, you need to create an ssl certificate, by doing the following (thanks to AJJ):

cd /System/Library/OpenSSL/certs sudo openssl req -new -x509 -nodes -days 365 -out imapd.pem -keyout imapd.pem

Fill in the fields with whatever values make most sense, but put "localhost" in the "Common Name" field.

Now you just need to create a new account in mail.app. Specify localhost as your server, and remember to alter the prefix path, so that imap does not try to show you your whole folder hierachy. If you store your mail in the folder ~/mail/, the prefix should be specified as mail/.

Warning: if you do not set a path, spotlight will end up trying to reindex all of your files, and spotlight will be very slow. If this happens, quit Mail and go into ~/Library/Mail. There will be a folder beginning "imap-XXXXXX" where XXXXXX is the name of your account. The cache is stored in that folder, in another folder called "Messages" which you should remove.

If using SSL, mail.app will complain when it first sees the certificate. To stop it from doing so (again thanks to AJJ):

Click "Show Certificate"
Alt-Drag the icon for the certificate onto your desktop.
Double-click the file.
In the dialog box that will appear, use the drop down list to add the key to the "X509Anchors" keychain.

Cleanup

You can now delete any files that you have put on your desktop during this process. You don't need them again.

Tips

I recommend that you go with the UW default of not hard-coding a path lower than your home dir, and specifying your mail folder in mail.app. There are no security benefits to not doing so, and you retain maximum flexibility.

All imapd options are set at compile time, there are no config files. This is a feature, apparently, and I can see the logic. See the docs that come with uw-imap for more details.

Mail.app is slightly flaky when handling IMAP servers, and I have found it useful to enable the "automatically syncronize changed folders" option [correction - doing this is not useful and slows down spotlight a little because you index the same stuff twice. I think]. When you first connect, mail.app will import your mail into a local cache and index it for spotlight searching. You can watch this happening with the "Activity Viewer" window. But it is very boring. Drink some tea and read a good book instead.

Trying to use mail.app while this indexing is happening is not a pleasant experience: mail.app may decide to show your folders as containing "no messages". In my experience, no damage is done: the mail is still there and will reappear after you quit and reload mail.

If you only want mail.app to see a subset of your mutt mail, why not create a folder (eg. ~/mail/export, and put symlinks to the folders, or files you want to see in mail.app there? Mail.app will follow symlinks and folder structures, so you can organise your email well, though the tools for doing this within mail are limited.