uva-fnwi / m365-imap Goto Github PK

You can't sign in here with a personal account. Use your work or school account instead.

Having used this script for a while to get the refresh token I got fed up with setting things in multiple places with my rather fragmented setup.

I wrote a basic script to be a single source of truth for all email providers. Subclassing Credentials just requires writing a refresh_token handler. All clients then use the access token. Refresh tokens are stored rather insecurely in files, but could easily be in a keyring.

If anyone is using multiple providers with expiring refresh tokens this might interest them. The office365 authenticator was written with an eye to this repo and ended up almost exactly the same (once I realised bottle 0.13 still isn't out...).

Many thanks for putting this up!

I was able to download mail using your instructions and offlineimap but I use Emacs to send email. Do you have a way to authenticate while sending email using Emacs' smtpmail-send-it if someone knows the following,

oauth2_client_id
oauth2_client_secret
oauth2_request_url
oauth2_refresh_token

I know that there is not a built-in method because smtpmail-auth-supported only has (cram-md5 plain login).

Hello,

Thank you very much for creating this repo. I was able to follow your instructions to use the Thunderbird app registration successfully with M365. Do you know if the same procedure could be used successfully with mbsync as well? Currently I am aware of the method here, though because the steps seem quite a bit more involved I am wondering your directions might somehow be reconciled with mbsync's configuration.

Again, thank you for the very helpful instructions here.

Hi!
First of all, thanks very much.

I've been using thunderbird with M365 tenant successfully, the tenant has forced modern authentication on, so i was hoping to swap to offlineimap.

For some reason, I keep getting the following error msg:

XOAUTH2 authentication failed: xoauth2handler got: {u'error_uri': u'https://login.microsoftonline.com/error?code=9002313', u'timestamp': u'2021-04-08 11:41:51Z', u'trace_id': u'fd30899c-96dc-44dd-ba5b-29c65c9db402', u'correlation_id': u'ec415584-c0d5-4149-b208-9c97edf767bd', u'error_description': u'AADSTS9002313: Invalid request. Request is malformed or invalid.\r\nTrace ID: fd30899c-96dc-44dd-ba5b-29c65c9db402\r\nCorrelation ID: ec415584-c0d5-4149-b208-9c97edf767bd\r\nTimestamp: 2021-04-08 11:41:51Z', u'error': u'invalid_grant', u'error_codes': [9002313]} PLAIN authentication failed: Connection is closed. 13 Enter password for user '[email protected]': LOGIN authentication failed: Connection is closed. 13 Failed to connect. Reason All authentication types failed:

Any idea what the source could be?

Hello again,

I use exim for my outgoing email, and it does not (yet?) support OAUTH natively.
I have now got it working with pretty much a 5 line gawk script, although there is a little more to it than that.

I started with exim4-oauth2 but that didn't work, and I don't understand how it could ever have worked, but adapted that approach for use with M365-IMAP.

You give instructions for msmtp. Would you like me to provide the instructions for exim as well?

In the course of getting exim to work with MS oauth, I came across a a statement from MS that refresh tokens expire after 90 days. You don't seem to allow for that: have I missed something?

I am planning to arrange to renew the refresh token every month or two in my gawk script, but
haven't done that yet.

Thanks for putting this together! Works like a charm. At my institution, I had to add this folderfilter to suppress lots of spurious errors:

[Repository Remote]
...
folderfilter = lambda folder: not folder.startswith('Calendar') and not folder.startswith('Contacts')

I figured this may help someone.

There is a new config.py option called Authority = "https://login.microsoftonline.com/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX/" which allows one to specify a tenantID. It works if provided. But this breaks older code that does not need a tenantID, and indeed, older versions of get_token.py and refresh_token.py do indeed work without a tenantID.

Shouldn't the code only use the tenantID if provided?

The tenantID/Authority option could also be better documented.

offlineimap returns

ERROR: Exceptions occurred during the run!
ERROR: HTTP Error 400: Bad Request (configuration is: {'client_id': '08162f7c-0fd2-4200-a84a-f25a4db0b584a', 'client_secret': 'TxRBilcHdC6WGBee]fs?QR:SJ8nI[g82', 'refresh_token': 'xxxxxxxx', 'grant_type': 'refresh_token'})

Everything was working fine until now

Question

Does the refresh token not change? If it does, would it need to be updated manually each time in offlineimap?

Suggestion

You should make the documentation link of offlineimap's ability to handle XOAUTH2 be this,

https://github.com/OfflineIMAP/offlineimap/blob/master/offlineimap.conf#L897

I have just upgraded a debian testing system, and offlineimap then fails to downoad from an
outlook.office365.com server with an error
ERROR: AttributeError: 'str' object has no attribute 'suppress_context'

The upgrade included the debian package python3-pyparsing version 3.1.1-1 which I suspect may be the problem.

=========================================================
Here are the errors including the backtrace:
Establishing connection to outlook.office365.com:993 (XXXXXXX)
ERROR: HTTP Error 400: Bad Request (configuration is: {'client_id': 'xxx', 'client_secret': 'xxxxxx', 'refresh_token': 'xxxxxxxx', 'grant_type': 'refresh_token'})
ERROR: While attempting to sync account 'xxxxxxx'
HTTP Error 400: Bad Request
*** Finished account 'xxxx' in 0:00
ERROR: Exceptions occurred during the run!
ERROR: 'str' object has no attribute 'suppress_context'
ERROR: Exceptions occurred during the run!
ERROR: While attempting to sync account 'xxxx'
urllib.error.HTTPError: HTTP Error 400: Bad Request

Traceback:
File "/usr/share/offlineimap3/offlineimap/accounts.py", line 298, in syncrunner
self.__sync()
File "/usr/share/offlineimap3/offlineimap/accounts.py", line 375, in __sync
remoterepos.getfolders()
File "/usr/share/offlineimap3/offlineimap/repository/IMAP.py", line 681, in getfolders
imapobj = self.imapserver.acquireconnection()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/share/offlineimap3/offlineimap/imapserver.py", line 593, in acquireconnection
self.__authn_helper(imapobj)
File "/usr/share/offlineimap3/offlineimap/imapserver.py", line 445, in __authn_helper
if func(imapobj):
^^^^^^^^^^^^^
File "/usr/share/offlineimap3/offlineimap/imapserver.py", line 379, in __authn_xoauth2
imapobj.authenticate('XOAUTH2', self.__xoauth2handler)
File "/usr/lib/python3/dist-packages/imaplib2.py", line 691, in authenticate
typ, dat = self._simple_command('AUTHENTICATE', mechanism.upper())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/imaplib2.py", line 1684, in _simple_command
return self._command_complete(self._command(name, *args), kw)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/imaplib2.py", line 1404, in _command
literal = literator(data, rqb)
^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/imaplib2.py", line 2247, in process
ret = self.mech(self.decode(data))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/share/offlineimap3/offlineimap/imapserver.py", line 249, in __xoauth2handler
response = urllib.request.urlopen(
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/urllib/request.py", line 216, in urlopen
return opener.open(url, data, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/urllib/request.py", line 525, in open
response = meth(req, response)
^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/urllib/request.py", line 634, in http_response
response = self.parent.error(
^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/urllib/request.py", line 563, in error
return self._call_chain(*args)
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/urllib/request.py", line 496, in _call_chain
result = func(*args)
^^^^^^^^^^^
File "/usr/lib/python3.11/urllib/request.py", line 643, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)

ERROR: AttributeError: 'str' object has no attribute 'suppress_context'
🧇

This project doesn't include a license. Adding a license or relicensing requires consent of all contributors, so it'd be good to add one before accepting more contributions. I recommend the MIT license.

I've set everything up correctly I think but how is the redirect_url supposed to direct the refresh_token back to the calling python script when there is no process open on port 5000?

1. The Thunderbird app client ID for Microsoft login changed recently. See https://searchfox.org/comm-central/source/mailnews/base/src/OAuth2Providers.jsm#143

2. "The redirect URI http://localhost:7598/ specified in the request does not match the redirect URIs configured for the application 9e5f94bc-e8a4-4e73-b8be-63364c29d753" Thunderbird app for Microsoft login now is set to use https in the redirect.

Firstly, thanks for making this. After following the steps in the README and running offlineimap, I get this error:

OfflineIMAP 7.3.0
  Licensed under the GNU GPL v2 or any later version (with an OpenSSL exception)
imaplib2 v3.06, Python v3.10.4, OpenSSL 1.1.1n  FIPS 15 Mar 2022
Account sync outlook:
 *** Processing account outlook
 Establishing connection to outlook.office365.com:993 (outlook-Remote)
 ERROR: While attempting to sync account 'outlook'
  LIST command error: BAD [b'User is authenticated but not connected.']. Data: b'CEDD6 LIST "" ""\r\n'
 *** Finished account 'outlook' in 0:04
ERROR: Exceptions occurred during the run!
ERROR: While attempting to sync account 'outlook'
  LIST command error: BAD [b'User is authenticated but not connected.']. Data: b'CEDD6 LIST "" ""\r\n'

Traceback:
  File "/usr/lib/python3.10/site-packages/offlineimap/accounts.py", line 298, in syncrunner
    self.__sync()
  File "/usr/lib/python3.10/site-packages/offlineimap/accounts.py", line 374, in __sync
    remoterepos.getfolders()
  File "/usr/lib/python3.10/site-packages/offlineimap/repository/IMAP.py", line 672, in getfolders
    imapobj = self.imapserver.acquireconnection()
  File "/usr/lib/python3.10/site-packages/offlineimap/imapserver.py", line 607, in acquireconnection
    listres = imapobj.list(self.reference, '""')[1]
  File "/usr/lib/python3.10/site-packages/offlineimap/imaplib2.py", line 917, in list
    return self._simple_command(name, directory, pattern, **kw)
  File "/usr/lib/python3.10/site-packages/offlineimap/imaplib2.py", line 1705, in _simple_command
    return self._command_complete(self._command(name, *args), kw)
  File "/usr/lib/python3.10/site-packages/offlineimap/imaplib2.py", line 1452, in _command_complete
    raise self.error('%s command error: %s %s. Data: %.100s' % (rqb.name, typ, dat, rqb.data))

Have you seen this error? Do you have any tips for resolving it? Here is my config:

[general]
accounts = outlook

[Account outlook]
localrepository = outlook-Local
remoterepository = outlook-Remote

[Repository outlook-Local]
type = Maildir
localfolders = ~/Maildir/Outlook

[Repository outlook-Remote]
type = IMAP
sslcacertfile = /etc/pki/tls/certs/ca-bundle.crt
remotehost = outlook.office365.com
remoteuser = <my email address>
# I tried both with and without specifying a password by adding and removing this line:
remotepass = <my password>
auth_mechanisms = XOAUTH2
oauth2_request_url = https://login.microsoftonline.com/common/oauth2/v2.0/token
oauth2_client_id = <thunderbird client_id>
oauth2_client_secret = <thunderbird client_secret>
oauth2_refresh_token = <contents of my imap_smtp_refresh_token file>

Any help would be appreciated!

The README assumes that one is familar with the term "tenant". I had to look it up which was not so easy given that it is such a general word. A link to
https://learn.microsoft.com/en-us/microsoft-365/solutions/tenant-management-overview?view=o365-worldwide
or some hint would have been helpful.

Thank you for the software.