isync

isync is a command line application to synchronize mailboxes; it supports Maildir and IMAP4 mailboxes. New messages, message deletions and flag changes can be propagated both ways.

Synchronization is based on unique message identifiers (UIDs), so no identification conflicts can occur (as opposed to some other mail synchronizers). Synchronization state is kept in one local text file per mailbox pair; multiple replicas of a mailbox can be maintained.

Note: isync is the name of the project, mbsync is the name of the executable

Installing

Install the isync package.

Configuring

Note: Google appears to block isync from downloading emails by default. If you have 2-step authentication enabled, you need to set up an app password and use that with isync, otherwise you need to go to Google's Security Page and toggle "Allow less secure apps" to "on".
Note: Subfolders setting in MaildirStore now seems to be required to be set: iSync Config SubFolders SubFolders Legacy worked as previous unset - Oct 2017

First create and customize the main configuration file using this example ~/.mbsyncrc:

~/.mbsyncrc
IMAPAccount gmail
# Address to connect to
Host imap.gmail.com
User username@gmail.com
Pass ***************
# To store the password in an encrypted file use PassCmd instead of Pass
# PassCmd "gpg2 -q --for-your-eyes-only --no-tty -d ~/.mailpass.gpg"
#
# Use SSL
SSLType IMAPS
# The following line should work. If you get certificate errors, uncomment the two following lines and read the "Troubleshooting" section.
CertificateFile /etc/ssl/certs/ca-certificates.crt
#CertificateFile ~/.cert/imap.gmail.com.pem
#CertificateFile ~/.cert/Equifax_Secure_CA.pem

IMAPStore gmail-remote
Account gmail

MaildirStore gmail-local
SubFolders Verbatim
# The trailing "/" is important
Path ~/.mail/gmail/
Inbox ~/.mail/gmail/Inbox

Channel gmail
Far :gmail-remote:
Near :gmail-local:
# Exclude everything under the internal [Gmail] folder, except the interesting folders
Patterns * ![Gmail]* "[Gmail]/Sent Mail" "[Gmail]/Starred" "[Gmail]/All Mail"
# Or include everything
#Patterns *
# Automatically create missing mailboxes, both locally and on the server
Create Both
# Sync the movement of messages between folders and deletions, add after making sure the sync works
Expunge Both
# Save the synchronization state files in the relevant directory
SyncState *

To get rid of the [Gmail]-Stuff (or [Google Mail] as in my case) in each mailbox name, it's possible to use separate Channels for each directory, and later merge them to a group:

~/.mbsyncrc
Channel sync-googlemail-default
Far :gmail-remote:
Near :gmail-local:
# Select some mailboxes to sync
Patterns "INBOX" "arch"

Channel sync-googlemail-sent
Far :gmail-remote:"[Google Mail]/Gesendet"
Near :gmail-local:sent
Create Near

Channel sync-googlemail-trash
Far :gmail-remote:"[Google Mail]/Papierkorb"
Near :gmail-local:trash
Create Near

# Get all the channels together into a group.
Group googlemail
Channel sync-googlemail-default
Channel sync-googlemail-sent
Channel sync-googlemail-trash

As you can see, name-translations are possible this way, as well.

Usage

First make any folders that were specified as Maildirs.

$ mkdir -p ~/.mail/gmail

Then to retrieve the mail for a specific channel run:

$ mbsync gmail

or to retrive the mail for all channels:

$ mbsync -a

Tips and tricks

Using Path and/or Inbox on NTFS partitions

Since ntfs partitions will not accept ; in a filename, you need to change your InfoDelimiter and your FieldDelimiter to something else, you can achieve this by globaly (outside any store or channel configuration) changing the later, like below:

~/.mbsyncrc
FieldDelimiter -

With a timer

If you want to automatically synchronize your mailboxes, isync can be started automatically with a systemd/User unit. The following service file can start the mbsync command:

~/.config/systemd/user/mbsync.service
[Unit]
Description=Mailbox synchronization service

[Service]
Type=oneshot
ExecStart=/usr/bin/mbsync -Va

[Install]
WantedBy=default.target

The following timer configures mbsync to be started 2 minutes after boot, and then every 5 minutes:

Once those two files are created, reload systemd, then enable and start , adding the flag to .

Tip: The mbsync service now only runs after login. It's also possible to launch the systemd-user instances after boot if you configure Systemd/User#Automatic start-up of systemd user instances.
Integration with notmuch or mu4e

If you want to run notmuch or mu/mu4e after automatically synchronizing your mails, it is preferable to modify the above mbsync.service by adding a post-start hook, like below:

You can also index by changing the line to , or to if you are running emacsclient and would like to index .

This modification assumes that you have already setup notmuch or mu/mu4e for your user. If the ExecStart command does not execute successfully, the ExecStartPost command will not execute, so be aware of this!

With imapnotify

IMAP IDLE is a way to get push notifications to download new email, rather than polling the server intermittently. This has the advantage of saving bandwidth and delivering your mail as soon as it's available. Isync does not have native IDLE support, but we can use a program like imapnotify to call mbsync when you receive new email. For this example we will use the package which is reported to work better with frequent network interruptions.

Install and create a configuration file for each mail server you want to poll. Note that the file name format, including the .conf, is necessary if you want to use the provided systemd service: (You can view the full configuration options in the project's README.)

Start/enable the user unit.

Note that IMAP IDLE only triggers when new mail arrives, not when there is undownloaded mail on the server. For example, if you receive 100 emails with your computer powered off, then turn on your computer, imapnotify will still not download new mail until you receive another email. For this reason you may want to run mbsync once when you log in.

Using XOAUTH2

First install an XOAUTH2 SASL plugin, like cyrus-sasl-xoauth2-gitAUR.

With oauth2token

Install and follow its README to configure the account. It will be responsible for getting the current XOAUTH2 token using the account credentials every time mbsync needs to authenticate.

Finally, add and , substituting <provider> and with the values you used for , to the section in the .

With mutt_oauth2.py

supports Google and Microsoft accounts out-of-the-box (for other cloud mail providers, you will need to provide their registration information). Download mutt_oauth2.py (make it executable and put in your PATH if desired) and follow its README for background and to configure the account.

In short, you will need to modify the and for your preferred encryption system as well as add the and from an app registration with the provider (see Isync#App registrations). Then to initialize your tokens, you run and answer the questions in:

$ mutt_oauth2.py userid@myschool.edu.tokens --verbose --authorize

To subsequently receive your access token (the script automatically handles renewing of tokens), you can run:

$ mutt_oauth2.py userid@myschool.edu.tokens

Finally, add and , with the correct paths if necessary, to the section in the .

App registrations

In order to get a and , you will need an app registration with the provider. You should create your own app registration if possible, otherwise you can use existing app registrations, such as Thunderbird's (under ), which are publicly available until dynamic client registration is supported.

Troubleshooting

SSL error

If you get the following error:

Since google enforce SNI when you use TLS 1.3, ensure to run at least isync v1.3.0 See https://sourceforge.net/p/isync/isync/merge-requests/2/ for more details

If you get certificate related errors like

SSL error connecting pop.mail.com (193.222.111.111:143): error:00000012:lib(0):func(0):reason(18) 

you may need to retrieve the server's certificates manually in order for mbsync to correctly verify it.

Step #1: Get the certificates

$ mkdir ~/.cert
$ openssl s_client -connect some.imap.server:port -showcerts 2>&1 < /dev/null | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' | sed -ne '1,/-END CERTIFICATE-/p' > ~/.cert/some.imap.server.pem

This will create a certificate file called (e.g. ). Alternatively one can download get_certs.sh and run it:

If you wish to do this manually, you may enter:

and it will display output something like:

Simply copy the first block that begins with and ends with , paste into a file, and save with a .pem extension (this is necessary for the next step). Older instructions state that, with Gmail, both certificate blocks must be saved but on testing this was found to be unnecessary.

Now, copy the root issuer certificate to your local certificate folder. In this example (Gmail), the root issuer is Equifax Secure Certificate Authority. This certificate is included in the package.

Step #2: Setup mbsync

Configure mbsync to use that certificate:

~/.mbsyncrc
IMAPAccount gmail
Host imap.gmail.com
# ...
CertificateFile ~/.cert/imap.gmail.com.pem

BAD Command with Exchange 2003

When connecting to an MS Exchange 2003 server, there could be problems when using pipelining (i.e. executing multiple imap commands concurrently). Such an issue could look as follows: So command 9 is to select a new folder, command 10 checks the mail and commands 11, 12 and 13 run in parallel, writing/getting/flagging a mail. In this case, the Exchange server would terminate the connection after the BAD return value and go on to the next channel. (And if all went well in this channel, mbsync would return with 0.) After setting

PipelineDepth 1

in the IMAPStore config part of the Exchange, this problem did not occur any more.

Emails on remote server have the wrong date

This fix works when syncing with fastmail, but it likely applies to other services as well.

If you move an email to a new folder using an email client, and mbsync causes the email to appear with the wrong date on the server, add this to your configuration file:

 CopyArrivalDate yes

For example, without this setting, moving an old email from Inbox to Archive using mu4e and then syncing to fastmail with mbsync will cause the email to appear in Archive but with the date of the sync.

mbsync uses mtime of email message when uploading from maildir to imap server. You can use fix_maildir_mail_mtime.py script to set mtime from email header.

This article is issued from Archlinux. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.