Notes on migrating to OAUTH2 with Gnus

The easiest way to configure a running Gnus instance is to use the *Server* buffer. From the *Group* buffer, one types ^ to enter the *Server* buffer. Once there, you will be able to add new server by pressing a. That will prompt for the server method (nnimap in my case), then a name (DavMail for me), then it opens up a new buffer with some cryptic code like

(nnimap "DavMail")

To enable Gnus to use davmail, I added:

(nnimap "DavMail"
	(nnimap-authinfo-file "~/.authinfo.gpg")
	(nnimap-address "localhost")
	(nnimap-server-port 1143)
	(nnimap-record-commands t)
	(nnimap-stream plain))

To explain:

1. The first line tells Gnus the server protocol and the name it will use;
2. the second line says to use the file .authinfo.gpg to get account information (username, password, hostname);
3. the third line says that the server's address is localhost;
4. the fourth line is the port the SMTP server is listening on (the default for davmail);
5. the fifth line is optional and really only useful for debugging;
6. the sixth line says to communicate with the server in plain text.

I can set-up an ssl connection between Gnus and davmail, but to get things working, I want to keep it simple.

As noted, Gnus will use authinfo.gpg to get password data from. So, I update it with the appropriate information:

machine localhost login UMnetid@umanitoba.ca password <mypassword>

Because localhost is the hostname of both IMAP and SMTP servers, I leave off any port information. To flush the stale authinfo data in Emacs, one does M-: (auth-source-forget-all-cached) RET.

The configuration file of Gnus needs updating, too. To gnus-posting-styles, I add:

("DavMail:.+"
 (address ,(cdr (assoc 'umanitoba email-addresses)))
 (X-Message-SMTP-Method "smtp localhost 1025 UMnetid@umanitoba.ca"))

This will ensure that emails in buffers with names starting with DavMail: will have email sent through davmail.

I use Debian testing, a Linux distribution, and unless I really need to do otherwise, I install software from Debian's repositories. So, installation was as simple as looking up the software and seeing what else needed to be installed.

root@t14:~# apt-cache search davmail
davmail - POP/IMAP/SMTP/CalDav/LDAP to Microsoft Exchange gateway

root@t14:~# apt-get -s install davmail

That lists a lot of extra software, but without email access, I can't do my job. So, I installed davmail:

root@t14:~# apt-get install davmail

See Linux setup for more information.

As it turned out, almost all the third-party documentation on davmail that I read is inaccurate. In fact, the most useful source of information is the DavMail FAQ and Getting Started pages.

Here are the settings that I eventually discovered work:

To test out my configuration, I navigate to the *Server* buffer in Emacs, place point over the DavMail server entry and type O to open the server. Davmail pops up a small window with instructions to click on the Office365 link. Doing so makes my default browser, firefox, open a new tab where I login to Office365 using Duo's MFA. Until the end of the login, the process appears to be the same as if I were logging into web-mail. However, the login redirects to a blank webpage. I copy that page's url and paste that back into the waiting davmail window, then click Send.

It works!

DavMail begins to index my mail while Gnus waits. I have read that this step can be slow, so I go do some other things and when I return, I have working email.

And yes, this was more like my tenth or fifteenth run, not my first.

The page https://blog.binchen.org/posts/how-to-get-email-from-exchange-server-without-outlook.html is out-of-date and should be avoided.

This thread was useful in alerting me to performance issues with davmail. But, the settings that are posted there (where clientid is set), contributed to my thinking that I should be using the clientid of my employer. That was a mistake.

This webpage seemed like it was matched perfectly to my situation (UVa uses Duo to do MFA, too). So, I had to read my log files very closely to realize that the advice to set the clientid was wrong for my needs.

The related page is out-of-date, too, since Thunderbird no longer needs to use a shim like davmail.

Roland Winkler initiated a long discussion on the Emacs devel mailing list in August 2021. The discussion eventually petered out without leading to any conclusions, but it is informative.

The Thunderbird configuration on Northwestern's site is correct and enabled me to painlessly configure Thunderbird.

In the Advanced tab (see 4.6.5), tick the box Use Exchange message size in IMAP. If this is not checked, then davmail reads every message to determine its size; if it is checked, then davmail only needs to read the Exchange header for each message. This speeds things up quite a bit.

Eventually, even with the speed-up mentioned, I gave up trying to use davmail to read email (imap). I have since set-up a paid account on a third-party email provider that does not use oauth2 and forward my email to it. I still use davmail to send outgoing mail through Exchange. This meant changing the gnus-posting-styles (see 4.4).

Admittedly, this remains a somewhat painful experience, because Gnus start-up time is still quite long due to the davmail (and Exchange) bottleneck. But, after that, the experience is much like using Gmail with Gnus.