gmail oauth2 authorization

[root]

I have been trying to authorize fetchmail via popserver.

The instructions provided by google for oauth2 authorization simply do not
work. My best guess is that they did work once - while password authorization
was possible - but now you have to get into your gmail account to authorize
your gmail account.

It has taken me 7.5 hours to confirm this.

For those interested you should read:

http://mmogilvi.users.sourceforge.net/software/oauthbearer.html

[Chris Vine]

On Thu, 2 Jun 2022 20:41:34 -0000 (UTC)

Post by root
I have been trying to authorize fetchmail via popserver.
The instructions provided by google for oauth2 authorization simply do not
work. My best guess is that they did work once - while password authorization
was possible - but now you have to get into your gmail account to authorize
your gmail account.
It has taken me 7.5 hours to confirm this.
http://mmogilvi.users.sourceforge.net/software/oauthbearer.html

Go to your google settings, choose 2 factor authentication and then get
an app password. Enter that app password in your fetchmailrc file.

[Rinaldi]

Post by Chris Vine
On Thu, 2 Jun 2022 20:41:34 -0000 (UTC)

Post by root
I have been trying to authorize fetchmail via popserver.
The instructions provided by google for oauth2 authorization simply do not
work. My best guess is that they did work once - while password authorization
was possible - but now you have to get into your gmail account to authorize
your gmail account.
It has taken me 7.5 hours to confirm this.
http://mmogilvi.users.sourceforge.net/software/oauthbearer.html

Go to your google settings, choose 2 factor authentication and then get
an app password. Enter that app password in your fetchmailrc file.

I can confirm this works. Seemed rather silly when I was doing it.

.fetchmailrc stanza:

poll pop.gmail.com with proto POP3 service 995
user '$USER' there with password '$PASSWD' is 'me' here ssl

rinaldi

[root]

Post by Rinaldi
I can confirm this works. Seemed rather silly when I was doing it.
poll pop.gmail.com with proto POP3 service 995
user '$USER' there with password '$PASSWD' is 'me' here ssl
rinaldi

Thanks rinaldi, that's what I had before the oauth2 was enabled.
It no longer works for me.

[Bit Twister]

Post by root

Post by Rinaldi
I can confirm this works. Seemed rather silly when I was doing it.
poll pop.gmail.com with proto POP3 service 995
user '$USER' there with password '$PASSWD' is 'me' here ssl
rinaldi

Thanks rinaldi, that's what I had before the oauth2 was enabled.
It no longer works for me.

Went through the google auth procedure to get app password.
logged into my google email account, brought to settings, show all
and set google mail to use imap. Configured it to do whatever the
client says to do with mail message.

installed dovecot on my system so I can use imap in claws-mail and
thunderbird apps. I did have to change my old google email login password
to new google app password.

Works for me. I did change from pop to imap.
# cat ~just_me/.fetchmailrc
#***************************************
# /accounts/just_me/.fetchmailrc
#***************************************

#********************
#* get any ISP email
#********************
poll "imap.gmail.com" with proto IMAP port 993
user "***@gmail.com" there with password "16_digit_google_app_pw_here"
is just_me here
options
ssl # download "seen" and "unseen" messages
fetchall # retrieve old and new messages
stripcr # Strip carriage returns from ends of lines
nokeep # delete new messages after retrieval


#****************************
#* get any credit card email
#****************************

poll "imap.gmail.com" with proto IMAP port 993
user "***@gmail.com" there with password "16_digit_google_app_pw_here"
is just_me here
options
ssl # download "seen" and "unseen" messages
fetchall # retrieve old and new messages
stripcr # Strip carriage returns from ends of lines
nokeep # delete new messages after retrieval


#*********** end accounts/hotmail/.fetchmailrc *******************

Did all the above for my
# grep mail /etc/passwd | wc -l
7
email accounts each of which runs fetchmail cron job hourly.

I also have a root cron which checks all linux mail boxs and
uses xmessage to tell me who needs to read any new mail.

With this setup there is no reason to log into gmail's email website.

[root]

Post by Rinaldi
poll pop.gmail.com with proto POP3 service 995
user '$USER' there with password '$PASSWD' is 'me' here ssl
rinaldi

Rinaldi please note:
the oauth2 was supposed to start a few days ago. My name is
early in the alphabet and I was hit this morning. It may be
that your account, while now active, may be shut down soon.

To be on the safe side, try to enable oauth2 now while
you still have working account.

[Chris Vine]

On Thu, 2 Jun 2022 22:24:29 -0000 (UTC)

Post by root

Post by Rinaldi
poll pop.gmail.com with proto POP3 service 995
user '$USER' there with password '$PASSWD' is 'me' here ssl
rinaldi

the oauth2 was supposed to start a few days ago. My name is
early in the alphabet and I was hit this morning. It may be
that your account, while now active, may be shut down soon.
To be on the safe side, try to enable oauth2 now while
you still have working account.

Nonsense. Instead of encouraging others to go down your dead end
which you say you have spent 7.5 hours on, accept that what people have
told you is right and get yourself an app password. You will not get
fetchmail to work any more with gmail without it. If you want true 2
factor authentication then you need to use a client like evolution or
thunderbird, which support it.

[root]

Post by Chris Vine
On Thu, 2 Jun 2022 22:24:29 -0000 (UTC)

Post by root

Post by Rinaldi
poll pop.gmail.com with proto POP3 service 995
user '$USER' there with password '$PASSWD' is 'me' here ssl
rinaldi

the oauth2 was supposed to start a few days ago. My name is
early in the alphabet and I was hit this morning. It may be
that your account, while now active, may be shut down soon.
To be on the safe side, try to enable oauth2 now while
you still have working account.

Nonsense. Instead of encouraging others to go down your dead end
which you say you have spent 7.5 hours on, accept that what people have
told you is right and get yourself an app password. You will not get
fetchmail to work any more with gmail without it. If you want true 2
factor authentication then you need to use a client like evolution or
thunderbird, which support it.

I was not saying that anything I did was right. I am saying that
yesterday Rinaldi's pop entry worked for me. Today it did not.
If you follow Google's instructions, you get to a point where
you can't access your account.

I only urged Rinaldi to get the oauth2 work done now.

I say here that your instructions for oauth2 DO WORK for
fetchmail.

Maybe you can show me your line in .mailrc which includes
the new 16 character password?

[root]

Post by root

Post by Chris Vine
On Thu, 2 Jun 2022 22:24:29 -0000 (UTC)

Post by root

Post by Rinaldi
poll pop.gmail.com with proto POP3 service 995
user '$USER' there with password '$PASSWD' is 'me' here ssl
rinaldi

the oauth2 was supposed to start a few days ago. My name is
early in the alphabet and I was hit this morning. It may be
that your account, while now active, may be shut down soon.
To be on the safe side, try to enable oauth2 now while
you still have working account.

Nonsense. Instead of encouraging others to go down your dead end
which you say you have spent 7.5 hours on, accept that what people have
told you is right and get yourself an app password. You will not get
fetchmail to work any more with gmail without it. If you want true 2
factor authentication then you need to use a client like evolution or
thunderbird, which support it.

I was not saying that anything I did was right. I am saying that
yesterday Rinaldi's pop entry worked for me. Today it did not.
If you follow Google's instructions, you get to a point where
you can't access your account.
I only urged Rinaldi to get the oauth2 work done now.
I say here that your instructions for oauth2 DO WORK for
fetchmail.
Maybe you can show me your line in .mailrc which includes
the new 16 character password?

OK, all is well. Whereas the password with four strings
of four characters each separated by spaces works for
fetchmail, the spaces must be eliminated in .mailrc.

Just another google POS.

[Bit Twister]

Post by root

Post by root

Post by Chris Vine
On Thu, 2 Jun 2022 22:24:29 -0000 (UTC)

Post by root

Post by Rinaldi
poll pop.gmail.com with proto POP3 service 995
user '$USER' there with password '$PASSWD' is 'me' here ssl
rinaldi

the oauth2 was supposed to start a few days ago. My name is
early in the alphabet and I was hit this morning. It may be
that your account, while now active, may be shut down soon.
To be on the safe side, try to enable oauth2 now while
you still have working account.

Nonsense. Instead of encouraging others to go down your dead end
which you say you have spent 7.5 hours on, accept that what people have
told you is right and get yourself an app password. You will not get
fetchmail to work any more with gmail without it. If you want true 2
factor authentication then you need to use a client like evolution or
thunderbird, which support it.

I was not saying that anything I did was right. I am saying that
yesterday Rinaldi's pop entry worked for me. Today it did not.
If you follow Google's instructions, you get to a point where
you can't access your account.
I only urged Rinaldi to get the oauth2 work done now.
I say here that your instructions for oauth2 DO WORK for
fetchmail.
Maybe you can show me your line in .mailrc which includes
the new 16 character password?

OK, all is well. Whereas the password with four strings
of four characters each separated by spaces works for
fetchmail, the spaces must be eliminated in .mailrc.
Just another google POS.

Weird, when google gave me the app pw, I just pasted it into files
needing it. None of which have spaces.

Yours might have worked if password was enclosed with/in quotes.

[root]

Post by Bit Twister
Weird, when google gave me the app pw, I just pasted it into files
needing it. None of which have spaces.
Yours might have worked if password was enclosed with/in quotes.

Nope, I tried that. My wife found out that when she cut and
pasted the spaces disappeared. I just typed them as they
were shown.

[root]

Post by Chris Vine
On Thu, 2 Jun 2022 20:41:34 -0000 (UTC)

Post by root
I have been trying to authorize fetchmail via popserver.

Go to your google settings, choose 2 factor authentication and then get
an app password. Enter that app password in your fetchmailrc file.

Could you please be more specific: I have chose 2 factor authorization,
I have enabled it on my android phone. What I don't follow is
"get an app password".

BTW, during the process of enabling the android phone, it
says you can skip this on reliable devices (such as your computer)
but that is not true.

Thanks.

[Chris Vine]

On Thu, 2 Jun 2022 21:51:48 -0000 (UTC)

Post by root

Post by Chris Vine
On Thu, 2 Jun 2022 20:41:34 -0000 (UTC)

Post by root
I have been trying to authorize fetchmail via popserver.

Go to your google settings, choose 2 factor authentication and then get
an app password. Enter that app password in your fetchmailrc file.

Could you please be more specific: I have chose 2 factor authorization,
I have enabled it on my android phone. What I don't follow is
"get an app password".
BTW, during the process of enabling the android phone, it
says you can skip this on reliable devices (such as your computer)
but that is not true.

Start up your browser on your slackware computer and go to google, click
on "Manage your google account", click on "Security", make sure you
hve selected 2 factor authentication and look for the "App passwords"
entry, choose a name (which can be anything and is just something to
identify it for you) and generate a password. I use the same app
password for fetchmail, postmail and mailx.

I can't say what you have done wrong and your last paragraph makes
no sense. Just relax. It looks as if you are working yourself up.

[Chris Vine]

On Fri, 3 Jun 2022 00:21:03 +0100

Post by Chris Vine
On Thu, 2 Jun 2022 21:51:48 -0000 (UTC)

Post by root

Post by Chris Vine
On Thu, 2 Jun 2022 20:41:34 -0000 (UTC)

Post by root
I have been trying to authorize fetchmail via popserver.

Go to your google settings, choose 2 factor authentication and then get
an app password. Enter that app password in your fetchmailrc file.

Could you please be more specific: I have chose 2 factor authorization,
I have enabled it on my android phone. What I don't follow is
"get an app password".
BTW, during the process of enabling the android phone, it
says you can skip this on reliable devices (such as your computer)
but that is not true.

Start up your browser on your slackware computer and go to google, click
on "Manage your google account", click on "Security", make sure you
hve selected 2 factor authentication and look for the "App passwords"
entry, choose a name (which can be anything and is just something to
identify it for you) and generate a password. I use the same app
password for fetchmail, postmail and mailx.

^^^^^^^^
postfix

[root]

Post by Chris Vine
On Thu, 2 Jun 2022 21:51:48 -0000 (UTC)

Post by root

Post by Chris Vine
On Thu, 2 Jun 2022 20:41:34 -0000 (UTC)

Post by root
I have been trying to authorize fetchmail via popserver.

Go to your google settings, choose 2 factor authentication and then get
an app password. Enter that app password in your fetchmailrc file.

Could you please be more specific: I have chose 2 factor authorization,
I have enabled it on my android phone. What I don't follow is
"get an app password".
BTW, during the process of enabling the android phone, it
says you can skip this on reliable devices (such as your computer)
but that is not true.

Start up your browser on your slackware computer and go to google, click
on "Manage your google account", click on "Security", make sure you
hve selected 2 factor authentication and look for the "App passwords"
entry, choose a name (which can be anything and is just something to
identify it for you) and generate a password. I use the same app
password for fetchmail, postmail and mailx.
I can't say what you have done wrong and your last paragraph makes
no sense. Just relax. It looks as if you are working yourself up.

FIXED fetchmail. Many thanks for that help, but mailx still does
not work.

When I got the Generated app password (set of four 4 character strings)
there was an email entry: ***@gmail.com with a dotted out
password. Did that have meaning?

Mailx is not fetchmail, does it require its own app password.

What makes me ask is that the password entry in .mailrc
set mta=smtps://MYNAME:***@smtp.gmail.com:465

where the server and password is set for mailx doesn't work
with either old or new entries.

[Bit Twister]

Post by root

Post by Chris Vine
Start up your browser on your slackware computer and go to google, click
on "Manage your google account", click on "Security", make sure you
hve selected 2 factor authentication and look for the "App passwords"
entry, choose a name (which can be anything and is just something to
identify it for you) and generate a password. I use the same app
password for fetchmail, postmail and mailx.
I can't say what you have done wrong and your last paragraph makes
no sense. Just relax. It looks as if you are working yourself up.

FIXED fetchmail. Many thanks for that help, but mailx still does
not work.

As I misunderstand it I thought mailx just reads your local system mail
box file and has nothing to do with outside the system mail.

Currently fetchmail sucks down any email and automagically sends to
my local username account mail and will be in /var/mail/local_user_login_here
.
easy enough to test for local mail for me by doing a
mail -s "local testshot" $LOGNAME < /dev/null
/var/mail/will have the testshot message and mail

I run Mageia Release 8 amd mailx is linked to mail
$ ls -l /usr/bin/mailx
lrwxrwxrwx 1 root root 14 Feb 13 2020 /usr/bin/mailx -> ../../bin/mail

Post by root
When I got the Generated app password (set of four 4 character strings)
password. Did that have meaning?
Mailx is not fetchmail, does it require its own app password.
What makes me ask is that the password entry in .mailrc

Cannot help there I have no ~/.mailrc
going to guess the set mta command would be used to get user mail from
another system.

Again. fetchmail pulls down your email from gmail.com and should be in
your local mailbox. No need for the .mailrc to also use gmail.com

And Yes. any "insecure" app will have to provide you gamial id and 16 digit
application gmail password to access its mail server.

Post by root
where the server and password is set for mailx doesn't work
with either old or new entries.

[root]

Post by Bit Twister

Post by root
FIXED fetchmail. Many thanks for that help, but mailx still does
not work.

As I misunderstand it I thought mailx just reads your local system mail
box file and has nothing to do with outside the system mail.

For me .mailrc configures my outgoing mail.
fetchmail works for incoming mail.

Post by Bit Twister
Currently fetchmail sucks down any email and automagically sends to
my local username account mail and will be in /var/mail/local_user_login_here

Yes, that is what happens for me.

Post by Bit Twister
.
easy enough to test for local mail for me by doing a
mail -s "local testshot" $LOGNAME < /dev/null
/var/mail/will have the testshot message and mail
I run Mageia Release 8 amd mailx is linked to mail
$ ls -l /usr/bin/mailx
lrwxrwxrwx 1 root root 14 Feb 13 2020 /usr/bin/mailx -> ../../bin/mail
Cannot help there I have no ~/.mailrc
going to guess the set mta command would be used to get user mail from
another system.
And Yes. any "insecure" app will have to provide you gamial id and 16 digit
application gmail password to access its mail server.

Chris Vine said the one password worked for fetchmail,postfix,and mailx.
I need a peek at the correct line in his .mailrc.

Thanks for responding BT.

[Bit Twister]

Post by root

Post by Bit Twister

Post by root
FIXED fetchmail. Many thanks for that help, but mailx still does
not work.

As I misunderstand it I thought mailx just reads your local system mail
box file and has nothing to do with outside the system mail.

For me .mailrc configures my outgoing mail.
fetchmail works for incoming mail.

HA ha. that would not work for me. I run a 3 node lan and batch jobs
send problems to me. gmail would not know my node's email address.

I use postfix as my MTA and if it cannot resolve the target address it
just forwards it to gmail.com from my node. For the other nodes I have
postfix configured to forward non-local email to my node.

Post by root
Chris Vine said the one password worked for fetchmail,postfix,and mailx.
I need a peek at the correct line in his .mailrc.

Going to guess Chris configured postfix to be able to do the secret smtp
handshake with gmail's smtp server.

[root]

Post by Bit Twister
Going to guess Chris configured postfix to be able to do the secret smtp
handshake with gmail's smtp server.

Chris can clarify that. I know nothing about postfix, never used it.

Thanks for responding.

[Bit Twister]

Post by root

Post by Bit Twister
Going to guess Chris configured postfix to be able to do the secret smtp
handshake with gmail's smtp server.

Chris can clarify that. I know nothing about postfix, never used it.

It comes installed configured to run locally. There you configure it by
hand to do whatever features you want enabled. For instance I have 7
user email accounts. If they send to anyone from the command line
I have configured postfix to change the from/reply address to their
gmail address then forward it to gmail.com.

For anyone curious there is https://www.postfix.org/documentation.html

[Chris Vine]

On Fri, 3 Jun 2022 02:21:05 -0000 (UTC)

Post by root

Post by Bit Twister
Going to guess Chris configured postfix to be able to do the secret smtp
handshake with gmail's smtp server.

Chris can clarify that. I know nothing about postfix, never used it.
Thanks for responding.

For postfix it is very much like mailx: the generated google password
becomes the sasl/tls password, and it doesn't actually use 2 factor
authentication. Because my postfix setup has a default relay which is
not smtp.gmail.com and only uses smtp.gmail.com for email coming from a
gmail address, I also use postfix's sender dependent authentication.
That won't be necessary if you always use gmail - just use the google
password as your sasl/tls password and set relayhost to
[smtp.gmail.com]:587

So my main.cf has amongst other things the following in it:

relayhost = my.default.relay:587
smtp_sasl_auth_enable = yes
smtp_sender_dependent_authentication = yes
sender_dependent_relayhost_maps = hash:/etc/postfix/sender_relay
smtp_tls_security_level = encrypt
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_sasl_security_options = noanonymous
smtp_sasl_tls_security_options = noanonymous

The sender_relay file has in it the relay for gmail mail, which takes
precedence over the default relay for mail with a ***@gmail.com from
address:

***@gmail.com [smtp.gmail.com]:587

The sasl_passwd file has in it the username and password for the
default relay and the gmail relay. In the case of gmail this is the 16
letter google generated one.

***@gmail.com ***@gmail.com:password
my.default.relay:587 username:password

[Jim Diamond]

On 2022-06-03 at 06:45 ADT, Chris Vine <***@cvine--nospam--.freeserve.co.uk> wrote:

<snip>

Post by Chris Vine
relayhost = my.default.relay:587
smtp_sasl_auth_enable = yes
smtp_sender_dependent_authentication = yes
sender_dependent_relayhost_maps = hash:/etc/postfix/sender_relay
smtp_tls_security_level = encrypt
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_sasl_security_options = noanonymous
smtp_sasl_tls_security_options = noanonymous

<snip>

Chris,

thanks for the details. I could not get outgoing email to gmail
without adding
smtp_sasl_mechanism_filter = login
to main.cf.

Just out of curiosity, do you already have that in your main.cf?

Thanks.
Jim

[Chris Vine]

On Mon, 6 Jun 2022 20:00:43 -0300

Post by Jim Diamond
<snip>

Post by Chris Vine
relayhost = my.default.relay:587
smtp_sasl_auth_enable = yes
smtp_sender_dependent_authentication = yes
sender_dependent_relayhost_maps = hash:/etc/postfix/sender_relay
smtp_tls_security_level = encrypt
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_sasl_security_options = noanonymous
smtp_sasl_tls_security_options = noanonymous

<snip>
Chris,
thanks for the details. I could not get outgoing email to gmail
without adding
smtp_sasl_mechanism_filter = login
to main.cf.
Just out of curiosity, do you already have that in your main.cf?

I have 'smtp_sasl_mechanism_filter = plain' in mine. I don't know
what the difference between 'plain' and 'login' SASL authentication is,
but from your results 'smtp_sasl_mechanism_filter = plain, login' may be
better.

[Jim Diamond]

Post by Chris Vine
On Mon, 6 Jun 2022 20:00:43 -0300

<snip>

Post by Chris Vine

Post by Jim Diamond
Chris,
thanks for the details. I could not get outgoing email to gmail
without adding
smtp_sasl_mechanism_filter = login
to main.cf.
Just out of curiosity, do you already have that in your main.cf?

I have 'smtp_sasl_mechanism_filter = plain' in mine. I don't know
what the difference between 'plain' and 'login' SASL authentication is,
but from your results 'smtp_sasl_mechanism_filter = plain, login' may be
better.

Thanks again. I see that just "plain" works for me as well. I guess
the default is something gmail finds unpalatable.

Jim

[andrew]

Post by Chris Vine
Start up your browser on your slackware computer and go to google, click
on "Manage your google account", click on "Security", make sure you
hve selected 2 factor authentication and look for the "App passwords"
entry, choose a name (which can be anything and is just something to
identify it for you) and generate a password. I use the same app
password for fetchmail, postmail and mailx.

You can add getmail and msmtp to that list, all works well here...

Andrew

[Poprocks]

Post by Chris Vine
On Thu, 2 Jun 2022 20:41:34 -0000 (UTC)

Post by root
I have been trying to authorize fetchmail via popserver.
The instructions provided by google for oauth2 authorization simply do not
work. My best guess is that they did work once - while password authorization
was possible - but now you have to get into your gmail account to authorize
your gmail account.
It has taken me 7.5 hours to confirm this.
http://mmogilvi.users.sourceforge.net/software/oauthbearer.html

Go to your google settings, choose 2 factor authentication and then get
an app password. Enter that app password in your fetchmailrc file.

Just wanted to say thanks for this -- here I was, thinking I'd have to
spend hours researching how to get mutt to cooperate with gmail after
the big recent changes. With app passwords it's very easy.