SMTP not working after upgrade to 3.8.1 from 3.7

stefano · November 22, 2022, 8:58pm

Checklist
I have read intro post: About the Installation Issues category
I have read the tutorials, help and searched for similar issues
I provide relevant information about my server (component names and versions, etc.)
I provide a copy of my logs and healthcheck
I describe the steps I have taken to trouble shoot the problem
I describe the steps on how to reproduce the issue

Hello,

We are using Passbolt CE on Digital Ocean since 1 year and it’s great.
We configured the email with Google SMTP relay and it worked since 3.8.1 update I did on 18th of Nov. Since now, I can only send test mails (successfully, by using the CLI cake script), but every mail triggered by the queue goes in SMTP timeout or SSL SMTP failure.

The system healthcheck script gives all OK: su -s /bin/bash -c "/usr/share/php/passbolt/bin/cake passbolt healthcheck --application --configFiles --core --database --environment --ssl" www-data

-------------------------------------------------------------------------------
 Healthcheck shell      
-------------------------------------------------------------------------------

 Environment

 [PASS] PHP version 8.1.12.
 [PASS] PCRE compiled with unicode support.
 [PASS] The temporary directory and its content are writable and not executable.
 [PASS] The logs directory and its content are writable.
 [PASS] GD or Imagick extension is installed.
 [PASS] Intl extension is installed.
 [PASS] Mbstring extension is installed.

 Config files

 [PASS] The application config file is present
 [PASS] The passbolt config file is present

 Core config

 [PASS] Debug mode is off.
 [PASS] Cache is working.
 [PASS] Unique value set for security.salt
 [PASS] Full base url is set to https://passbolt.gigadesignstudio.com
 [PASS] App.fullBaseUrl validation OK.
 [PASS] /healthcheck/status is reachable.

 SSL Certificate

 [PASS] SSL peer certificate validates
 [PASS] Hostname is matching in SSL certificate.
 [PASS] Not using a self-signed certificate

 Database

 [PASS] The application is able to connect to the database
 [PASS] 26 tables found
 [PASS] Some default content is present
 [PASS] The database schema up to date.

 Application configuration

 [PASS] Using latest passbolt version (3.8.1).
 [PASS] Passbolt is configured to force SSL use.
 [PASS] App.fullBaseUrl is set to HTTPS.
 [PASS] Selenium API endpoints are disabled.
 [PASS] Search engine robots are told not to index content.
 [PASS] Registration is closed, only administrators can add users.
 [PASS] Serving the compiled version of the javascript app
 [PASS] All email notifications will be sent.

 [PASS] No error found. Nice one sparky!

This command works: su -s /bin/bash -c "/usr/share/php/passbolt/bin/cake passbolt send_test_email --recipient=stefano@gigadesignstudio.com" www-data

-------------------------------------------------------------------------------
Email configuration
-------------------------------------------------------------------------------
Host: smtp-relay.gmail.com
Port: 587
Username: 
Password: *********
TLS: true
Sending email from: Giga password service <info@gigadesignstudio.com>
Sending email to: stefano@gigadesignstudio.com
-------------------------------------------------------------------------------
Trace
[220] smtp-relay.gmail.com ESMTP b15-20020a50cccf000000b00469701c73c7sm337560edj.47 - gsmtp
 EHLO 161.35.206.138
[250] smtp-relay.gmail.com at your service, [161.35.206.138]
[250] SIZE 157286400
[250] 8BITMIME
[250] STARTTLS
[250] ENHANCEDSTATUSCODES
[250] PIPELINING
[250] CHUNKING
[250] SMTPUTF8
 STARTTLS
[220] 2.0.0 Ready to start TLS
 EHLO 161.35.206.138
[250] smtp-relay.gmail.com at your service, [161.35.206.138]
[250] SIZE 157286400
[250] 8BITMIME
[250] AUTH LOGIN PLAIN XOAUTH2 PLAIN-CLIENTTOKEN OAUTHBEARER XOAUTH
[250] ENHANCEDSTATUSCODES
[250] PIPELINING
[250] CHUNKING
[250] SMTPUTF8
 MAIL FROM:<info@gigadesignstudio.com>
[250] 2.1.0 OK b15-20020a50cccf000000b00469701c73c7sm337560edj.47 - gsmtp
 RCPT TO:<stefano@gigadesignstudio.com>
[250] 2.1.5 OK b15-20020a50cccf000000b00469701c73c7sm337560edj.47 - gsmtp
 DATA
[354] Go ahead b15-20020a50cccf000000b00469701c73c7sm337560edj.47 - gsmtp
 From: Giga password service <info@gigadesignstudio.com>
To: stefano@gigadesignstudio.com
Date: Tue, 22 Nov 2022 20:37:01 +0000
Message-ID: <9a861794c73f4a068c6e25546fb07f42@passbolt-ubuntu-s-1vcpu-1gb-intel-fra1-01>
Subject: Passbolt test email
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Congratulations!
If you receive this email, it means that your passbolt smtp configuration is working fine.

.
[250] 2.0.0 OK  1669149421 b15-20020a50cccf000000b00469701c73c7sm337560edj.47 - gsmtp
 QUIT
The message has been successfully sent!

This command fails: su -s /bin/bash -c "/usr/share/php/passbolt/bin/cake EmailQueue.sender" www-data

su -s /bin/bash -c "/usr/share/php/passbolt/bin/cake EmailQueue.sender" www-data
SMTP server did not accept the connection or trying to connect to non TLS SMTP server using TLS.
Email 1910 was not sent

I’m attaching the screenshot of a query on the passbolt DB, you can see that after the update every email in the “email_queue” table fails after 4 retries (5th column); error is in the 6th column.

Thank you, have a nice day
Stefano

pabloelcolombiano · November 22, 2022, 9:19pm

Hi,

does the following command return any warnings or erros?

su -s /bin/bash -c "/usr/share/php/passbolt/bin/cake passbolt healthcheck --gpg --smtpSettings" www-data

stefano · November 23, 2022, 8:57am

Hi Pablo,

This is the result of the command… everything seems ok

-------------------------------------------------------------------------------
 Healthcheck shell  
-------------------------------------------------------------------------------

 GPG Configuration

 [PASS] PHP GPG Module is installed and loaded.
 [PASS] The environment variable GNUPGHOME is set to /var/lib/passbolt/.gnupg.
 [PASS] The directory /var/lib/passbolt/.gnupg containing the keyring is writable by the webserver user.
 [PASS] The server OpenPGP key is not the default one
 [PASS] The public key file is defined in config/passbolt.php and readable.
 [PASS] The private key file is defined in config/passbolt.php and readable.
 [PASS] The server key fingerprint matches the one defined in config/passbolt.php.
 [PASS] The server public key defined in the config/passbolt.php (or environment variables) is in the keyring.
 [PASS] There is a valid email id defined for the server key.
 [PASS] The public key can be used to encrypt a message.
 [PASS] The private key can be used to sign a message.
 [PASS] The public and private keys can be used to encrypt and sign a message.
 [PASS] The private key can be used to decrypt a message.
 [PASS] The private key can be used to decrypt and verify a message.
 [PASS] The public key can be used to verify a signature.
 [PASS] The server public key format is Gopengpg compatible.
 [PASS] The server private key format is Gopengpg compatible.

 SMTP Settings

 [PASS] The SMTP Settings plugin is enabled.
 [PASS] SMTP Settings coherent. You may send a test email to validate them.
 [PASS] The SMTP Settings source is: database.

 [PASS] No error found. Nice one sparky!

Stefano

garrett · November 25, 2022, 3:33pm

@stefano There are other users experiencing similar symptoms regarding email delivery problems following the upgrade. I don’t think it’s been determined yet what is causing this.

stefano · November 25, 2022, 3:45pm

Thank you,

We’ll wait for some feedbacks in the next days.

Bye

garrett · November 25, 2022, 3:55pm

@stefano are you seeing anything error-wise on the Gmail side of things? Or, any coincidental changes in the settings for SMTP such as IP whitelisting, etc?

cedric · November 30, 2022, 2:18pm

Hello @stefano,

We are currently working on a fix that could fix your issue.

In the meantime can you let me know if you are able to send the test email via the administration workspace directly from the application? Could you try to update the configuration from there also? It might unlock the situation if something went wrong during the migration.

max · November 30, 2022, 2:43pm

We will centralise the coms on this thread: Issue with Gnupg decryption in Cron job / Emails not sent
Thanks

stefano · December 1, 2022, 7:53am

Hello @cedric,

It works, I can send the test email from inside the configuration panel, Email Server tab.
Unfortunately, updating the configuration from there doesn’t affect the real notification emails.

Regards,

pabloelcolombiano · December 5, 2022, 4:16pm

Thank you @stefano for providing us additional information on your setup.

The issue is caused by the ‘client’ parameter in the SMTP Settings, which is not supported by the SMTP settings stored in the database so far.

How to detect if my passbolt is concerned:
In passbolt.php AND in app.php files, check if a client is set under the EmailTransport configuration.
If a client is defined, and this client is required, this should be the origin of the issue.
How to fix this until the next release
In this case only, otherwise please report your issue mentioning that the client is not defined in the settings on file.
Delete the SMTP settings in the database:
DELETE FROM organization_settings WHERE property = "smtp";

By relying on the configuration files, the SMTP settings should work again. We will ensure that client is supported for DB settings in a future release and will keep you posted.

abrakadabra · January 6, 2023, 11:08am

I have solved this via setting client to the actual IP address. This is not mentioned anywhere in the config instructions.

pabloelcolombiano · January 6, 2023, 12:06pm

Hi @abrakadabra ,

thank you for posting your solution here. We are working on a solution to include the client in the config setup. This should be available soon.

garrett · January 6, 2023, 6:12pm

@abrakadabra I’ve removed the link to the external article. You posted this link in 3 different threads. Please do not spam threads repeatedly with links to external articles, or with the same information. Thank you!