How to use Google G suite OAuth2 with mbsync (from isync) and msmtp on NetBSD

This post will be irrelevant for Google Workspace after 2022-10-04 because OAuth out-of-band (OBB) flow will be deprecated. See my newer post which describes loopback IP flow for mbsync and msmtp.

Google will remove Less Secure app Access (LSA) from Gmail of G Suite. And I want to continue to use Gmail from NetBSD laptop with mbsync (isync) and msmtp. However I cannot find any successful report about mbsync with XOAuth2.

I do not understand OAuth2 and its derivatives at all. This post is for reporting how to use mbsync and msmtp for Gmail without LSA, not for understanding OAuth2.

Prerequiestics

I am a NetBSD desktop user. I will use pkgsrc as a package management system. The pkgsrc is portable for almost all POSIX system. You can bootstrap pkgsrc on non-NetBSD POSIX-like operating system that you are using.

I should install the following pkgsrc packages.

  • pkgsrc/mail/isync (isync-1.3.1nb2 or later)
  • pkgsrc/security/cyrus-sasl-xoauth2
  • pkgsrc/mail/msmtp
  • Be sure to use isync-1.3.1nb2 or later. isync-1.3.1nb1 has short buffer (80 bytes) for an output of PassCmd.

    Generate client ID and client secret on Google CLoud Platform console

    Goole Cloud Platform console is very complicated. I will not show how to generate clinet ID and client secret step by step.

    I should follow Google Cloud Platform console -> API&Services -> Credentials -> OAuth 2.0 Client IDs.

    Get my Access Token and Refresh Token with Google's oauth2.py

    With my client ID and client secret, I can get my (temporal) Access Token and Refresh Token with Google's oauth2.py commandline tool. I can get oauth2.py tool from https://raw.githubusercontent.com/google/gmail-oauth2-tools/master/python/oauth2.py. This oauth2.py is for Python 2.7 not for Python 3. Be sure to execute the tool with python2.7 command from pkgsrc/lang/python27.

    And I must have modern web browser like pkgsrc/www/firefox to access Google's webpage.

    An outline of my steps is as follows.

    1. Execute oauth2.py with my client ID and client secret.
    2. I can get a URI of Google's webpage and open the webpage with my Firefox.
    3. I can get a code and input the code to oauth2.py.
    4. I can get the access token and refresh token.

    I will use the refresh token to renew the access token. Do not discard my refresh token.

    I will execute oauth2.py as follows.

    $ python2.7 /opt/share/oauth2.py \
    --user=MY_GSUITE_EMAIL_ADDRESS \
    --client_id=MY_CLIENT_ID.apps.googleusercontent.com \
    --client_secret=MY_CLIENT_SECRET  \
    --generate_oauth2_token
    

    Create a script to generate a access token from the refresh token

    My access token will expire in 3600 seconds. I will regenerate my access token every e-mail send/fetch. Google's oauth2.py has no special mode to output refresh access token only. I will create the following script as /opt/bin/get_teteraorg_token.sh.

    #! /bin/sh
    /usr/pkg/bin/python2.7 /opt/share/oauth2.py \
    --user=MY_GSUITE_EMAIL_ADDRESS \
    --client_id=MY_CLIENT_ID.apps.googleusercontent.com \
    --client_secret=MY_CLIENT_SECRET \
    --generate_oauth2_token \
    --refresh_token=MY_REFRESH_TOKEN | \
    awk -F" " '{if(NR==1)print $3}'
    

    Do not ferget to execute chmod 700 /opt/bin/get_teteraorg_token.sh.

    My ~/.mbsyncrc

    My .mbsyncrc is very simple. PassCmd and AuthMechs are special.

    IMAPAccount gmail
    Host imap.gmail.com
    User MY_GSUITE_EMAIL_ADDRESS
    #AuthMechs LOGIN
    AuthMechs XOAUTH2
    PassCmd "/opt/bin/get_teteraorg_token.sh"
    SSLType IMAPS
    CertificateFile /etc/openssl/certs/ca-certificates.crt
    
    IMAPStore gmail-remote
    Account gmail
    
    MaildirStore gmail-local
    Path ~/.maildir/teteraorg-gmail/
    Inbox ~/.maildir/teteraorg-gmail/inbox
    
    Channel gmail-inbox
    Master :gmail-remote:
    Slave :gmail-local:
    
    Channel gmail-trash
    Master :gmail-remote:"[Gmail]/Trash"
    Slave :gmail-local:trash
    
    Channel gmail-sent
    Master :gmail-remote:"[Gmail]/Sent Mail"
    Slave :gmail-local:sent
    
    Channel gmail-archive
    Master :gmail-remote:
    Slave :gmail-local:
    Patterns "Arch*"
    
    Group teteraorg
    Channel gmail-inbox
    Channel gmail-sent
    Channel gmail-trash
    Channel gmail-archive
    

    My ~/.msmtprc

    For msmtp, no special patch is reaquired. msmtp has no XOAuth2 support (it seems that XOauth2 support is removed already). Use auth oauthbearer instead.

    defaults
    
    account teteraorg
    tls on
    tls_certcheck off
    tls_starttls off
    host smtp.gmail.com
    port 465
    protocol smtp
    auth oauthbearer
    from MY_GSUITE_EMAIL_ADDRESS
    user MY_GSUITE_EMAIL_ADDRESS
    passwordeval "/opt/bin/get_teteraorg_token.sh"
    
    account default: teteraorg
    

    5 件のコメント:

    1. Why do you have three channels? The setups that I have seen largely do just

      Patterns * ![Gmail]* "[Gmail]/Sent Mail" "[Gmail]/Starred" "[Gmail]/All Mail"

      for a single channel

      返信削除
      返信
      1. Thank you. I will try to combine the channels.

        削除
    2. このコメントは投稿者によって削除されました。

      返信削除
    3. THanks for this instruction. I tried this but getting into this error.

      Error: SASL(-1): generic failure: Unable to find a callback: 18948

      Is there any way to fix this?

      According to https://sourceforge.net/p/isync/mailman/message/34411104/, we should use LOGIN instead of XOAUTH2, but I am not sure it's right.

      返信削除
      返信
      1. If you have enabled XOAUTH2 authentication on Google Workspace console, you will not be able to use LOGIN authmech. LOGIN authmech indicates userID/password authentication.

        Your environment is macOS? 18948 is 0x4a04. 0x4a04 is not defined in plain cyrus-sasl-2.1.27 for callback ID. Google search says 0x4a04 is defined for Apple version of sasl.h. See: https://opensource.apple.com/source/passwordserver_sasl/passwordserver_sasl-211/cyrus_sasl/include/sasl.h.auto.html . I have no macOS environment. Does macOS's Cyrus SASL support OAUTHBEARER or XOATH2 natively? If so, cyrus-sasl-xoauth2 may not be required.
        Thank you.

        削除

    注: コメントを投稿できるのは、このブログのメンバーだけです。

    MPEG 2.5

    MPEG 2.5と言う表記を見掛けたのだが、どういうものか理解できなかった。 /usr/pkgsrc/audio を以下のように検索すると、 /usr/pkgsrc/audio/libmad/DESCR に、MPEG-2 extension to Lower Sam...