Google will remove old OAuth out-of-band (OOB) support on 2022-10-04. And desktop clients for Google Workspace should use Loopback IP address flow instead. So I must change my script to support Loopback IP address flow. My previous setup using OOB is described in How to use Google G suite OAuth2 with mbsync (from isync) and msmtp on NetBSD post.
I have searched the web and found How to continue using msmtp OAuth 2.0 for Gmail in mutt after oob deprecation? on StackExchange. The answer is to use getmail-gmail-xoauth-tokens script from getmail6 to get refresh token and access tokens. The answer says "- which are what we've been looking for, but that 1h expiry is prohibitively awkward...". It is absolutely unacceptable for me. I am away from GUI or web browsers in most time. I have not read getmail-gmail-xoauth-tokens script. However I feel that it may be the answer's misunderstanding or misuse. Anyway I do not have enough time to investigate the potentially hopeless script.
As a result, I found that the access tokens will expire in 3600 seconds and the refresh token has much longer life time like before. I have create two scripts to confirm this result. And I can reuse the scripts for my mbsync from isync and msmtp setups.
Get refresh token and save it
At first, you must download your client secret file as client_secret_*.json. Just rename it as client_secret.json
Google provides Google Auth Python Library and related libraries for Python. For further changes, I should use common methods to get tokens. I am a pkgsrc user and I have installed required libraries as follows:
# cd /usr/pkgsrc/www/py-google-api-python-client # make install # cd /usr/pkgsrc/security/py-google-auth-oauthlib # make install
To get refresh token, you can use google_auth_oauthlib.flow.InstalledAppFlow.from_client_secrets_file() with the client_secret.json
file.
My source code (google-oauth2-loopback-ip.py) is as follows:
#!/usr/pkg/bin/python3.10
# Get refresh token and save it as ~/.refresh_token.pickle.
# Saved information will be used to refresh access token hourly.
# Follow:
# https://googleapis.github.io/google-api-python-client/docs/oauth-installed.html
import os
import pickle
from google_auth_oauthlib.flow import InstalledAppFlow
# Constants
CLIENT_SECRETS_FILE = 'client_secret.json'
CREDENTIAL_CACHE_FILE = '.refresh_token.pickle'
## Configurations
### To identify scopes and services, See:
### https://developers.google.com/identity/protocols/oauth2/scopes#gmail
SCOPES=['https://mail.google.com/']
API_SERVICE_NAME = 'mail'
API_VERSION = 'v1'
## Get credentials
def get_credentials():
## Create client object
flow = InstalledAppFlow.from_client_secrets_file(
CLIENT_SECRETS_FILE,
scopes=SCOPES)
## Open with local web browser, input credential manually
flow.run_local_server(host='localhost',
port=8091,
authorization_prompt_message='Please visit this URI: {url}',
success_message='The auth flow is complete; you may close this web browser window.',
open_browser=True)
return flow.credentials
if __name__ == '__main__':
credentials = get_credentials()
# Get home directory path and create a path for cache file
homeDir = os.environ['HOME']
if not homeDir:
homeDir = '.'
credPath = os.path.join(homeDir, CREDENTIAL_CACHE_FILE)
# Save credentials as file
with open(credPath, 'wb') as tokenCache:
pickle.dump(credentials, tokenCache)
# https://google-auth.readthedocs.io/en/stable/reference/google.oauth2.credentials.html#google.oauth2.credentials.Credentials
print('Refresh Token:', credentials.refresh_token)
You will get your refresh token as Refresh Token: YOURREFRESHTOKENSTRING.
This script get an access token simultaniously (See your ~/.refresh_token.pickle). However I will not use the access token in this script at all.
Get access token using the refresh token
The refresh token has longer life time than 3600 seconds (life time of the access token). And the refresh token is used to get refreshed access tokens without login via web browsers.
To get a refreshed access token, you should load saved credentials in ~/.refresh_token.pickle and invoke refresh() method.
My source code (google-oauth2-refresh-access_token.py) is as follows:
#!/usr/pkg/bin/python3.10
# Get new access token using saved refresh token.
# Follow:
# https://googleapis.github.io/google-api-python-client/docs/oauth-installed.html
import os
import pickle
from google.auth.transport.requests import Request
# Constants
CREDENTIAL_CACHE_FILE = '.refresh_token.pickle'
## Configurations
### To identify scopes and services, See:
### https://developers.google.com/identity/protocols/oauth2/scopes#gmail
SCOPES=['https://mail.google.com/']
API_SERVICE_NAME = 'mail'
API_VERSION = 'v1'
## Refresh credentials and get new ones
def get_refreshed_credentials():
# Get home directory path and create a path for cache file
homeDir = os.environ['HOME']
if not homeDir:
homeDir = '.'
credPath = os.path.join(homeDir, CREDENTIAL_CACHE_FILE)
if os.path.exists(credPath):
with open(credPath, 'rb') as tokenCache:
credentials = pickle.load(tokenCache)
if credentials:
# Refresh access token with refresh token
credentials.refresh(Request())
return credentials
if __name__ == '__main__':
credentials = get_refreshed_credentials()
# https://google-auth.readthedocs.io/en/stable/reference/google.oauth2.credentials.html#google.oauth2.credentials.Credentials
print('Access Token:', credentials.token)
#print('Expiry:', credentials.expiry) # in UTC
You will get your access token as Access Token: YOURREFRESHEDACCESSTOKENSTRING.
For mbsync and msmtp
This part is almost identical to my previous post. I will include my current script and configuration files.
$ cat /opt/bin/get_teteraorg_token.sh
#!/bin/sh
python3.10 /opt/bin/google-oauth2-refresh-access_token.py | awk -F" " '{if(NR==1)print $3}'
$ cat ~/.mbsync (snip) IMAPAccount gmail Host imap.gmail.com User username@tetera.org AuthMechs XOAUTH2 PassCmd "/opt/bin/get_teteraorg_token.sh" SSLType IMAPS CertificateFile /etc/openssl/certs/ca-certificates.crt (snip)
$ cat ~/.msmtprc (snip) account teteraorg tls on tls_certcheck off tls_starttls off host smtp.gmail.com port 465 protocol smtp auth xoauth2 from username@tetera.org user username@tetera.org passwordeval "/opt/bin/get_teteraorg_token.sh" (snip)
