Test/notification emails work but registration/recovery emails only rarely

These errors are because passbolt is attempting SMTP but we have found your exchange server is not needing/using/accepting SMTP at that port. Typically mail servers receive incoming mail on port 25, and do SMTP on port 587. Sometimes older servers do SMTP on port 25 but I wouldn’t consider it standard.

Passbolt using SMTP attempts an auth process, but that is not working either, because the remote port is not offering SMTP connections.

As we can send email straight to the remote port 25, we know it’s at least receiving incoming mail.

Hard to know what to recommend for you to do. It’s not passbolt I don’t think, or your settings…it’s the mail server and its settings.

Have you tried mail server port 587 (even though they say no credentials are needed, maybe that would work)?

“stream_socket_client(): Unable to connect to tcp://127.0.0.1:587 (Connection refused)” is what I get when switching the port - also the same for couple different IP’s behind the load balanced DNS name for the SMTP.

Thanks for all the help, I’ll talk to the Exchange team on Monday. I don’t even know what could be different about this Passbolt setup compared to any of the hundreds of different services/servers we have across the company that use the same SMTP. That is basically why I was locked down on the idea that its either something wrong with our Passbolt setup or Passbolt itself but those seem pretty unlikely right now too based on our discussion here.

1 Like

Discussed with the exchange team yesterday and they can’t see issues. For any email that does not get send does not seem to actually reach the exchange server. I asked them to look more thoroughly as this seemed very odd but logs they showed me only had emails that actually did come through.

As I’m pretty desperate with this I actually setup a new server for this where I migrated our DB. Original setup was on RedHat 8.7 with Passbolt version 3.8.0 - new one is on Ubuntu 22.04 with Passbolt 4.0.2. Sadly the same issue persists although I’m quite liking the few new features that have come along since our initial install!

One of my team members had an issue after install where his browser was just taking him to the registration page instead of logon. He noticed that when he did the registration twice in quick succession the email actually came through. I can see that on the email_queue table on the DB too that the first one failed and second one worked. This also seemingly followed the “every other email works” pattern we saw. Still don’t know if this is coincidence or not but it seems to have too much repetition to be totally random occurrence.

Below sample from the table - the cases where the “every other” pattern breaks so that there are multiple emails that fail to send 4 times in a row are all 100% cases where registration/recovery has been tried. Maybe this is one form of workaround we have now that we just spam the registration/recovery until an email comes through but seems very sad for a SW that works so well otherwise. Its also annoying that someone always misses the “edit” notifications.

mysql> SELECT email, sent, send_tries, send_at, created, error FROM email_queue ORDER BY id DESC LIMIT 40;
+----------------------------+------+------------+---------------------+---------------------+----------------------------------------------------------------------------+
| email                      | sent | send_tries | send_at             | created             | error                                                                      |
+----------------------------+------+------------+---------------------+---------------------+----------------------------------------------------------------------------+
| obscured@passbolt.com |    1 |          0 | 2023-05-30 12:06:47 | 2023-05-30 12:06:47 | NULL                                                                       |
| obscured@passbolt.com |    0 |          4 | 2023-05-30 12:05:41 | 2023-05-30 12:05:41 | SMTP authentication method not allowed, check if SMTP server requires TLS. |
| obscured@passbolt.com |    1 |          0 | 2023-05-30 11:50:32 | 2023-05-30 11:50:32 | NULL                                                                       |
| obscured@passbolt.com |    0 |          4 | 2023-05-30 11:48:10 | 2023-05-30 11:48:10 | SMTP authentication method not allowed, check if SMTP server requires TLS. |
| obscured@passbolt.com |    0 |          4 | 2023-05-30 11:42:24 | 2023-05-30 11:42:24 | SMTP authentication method not allowed, check if SMTP server requires TLS. |
| obscured@passbolt.com |    1 |          0 | 2023-05-29 11:42:50 | 2023-05-29 11:42:50 | NULL                                                                       |
| obscured@passbolt.com |    1 |          1 | 2023-05-29 11:42:50 | 2023-05-29 11:42:50 | SMTP authentication method not allowed, check if SMTP server requires TLS. |
| obscured@passbolt.com |    1 |          0 | 2023-05-29 11:42:50 | 2023-05-29 11:42:50 | NULL                                                                       |
| obscured@passbolt.com |    1 |          2 | 2023-05-29 11:42:50 | 2023-05-29 11:42:50 | SMTP authentication method not allowed, check if SMTP server requires TLS. |
| obscured@passbolt.com |    1 |          0 | 2023-05-29 11:42:42 | 2023-05-29 11:42:42 | NULL                                                                       |
| obscured@passbolt.com |    1 |          1 | 2023-05-29 11:42:42 | 2023-05-29 11:42:42 | SMTP authentication method not allowed, check if SMTP server requires TLS. |
| obscured@passbolt.com |    1 |          0 | 2023-05-29 11:42:42 | 2023-05-29 11:42:42 | NULL                                                                       |
| obscured@passbolt.com |    0 |          4 | 2023-05-29 11:42:42 | 2023-05-29 11:42:42 | SMTP authentication method not allowed, check if SMTP server requires TLS. |
| obscured@passbolt.com |    1 |          0 | 2023-05-29 11:42:29 | 2023-05-29 11:42:29 | NULL                                                                       |
| obscured@passbolt.com |    1 |          1 | 2023-05-29 11:42:29 | 2023-05-29 11:42:29 | SMTP authentication method not allowed, check if SMTP server requires TLS. |
| obscured@passbolt.com |    1 |          0 | 2023-05-29 11:42:29 | 2023-05-29 11:42:29 | NULL                                                                       |
| obscured@passbolt.com |    1 |          2 | 2023-05-29 11:42:29 | 2023-05-29 11:42:29 | SMTP authentication method not allowed, check if SMTP server requires TLS. |
| obscured@passbolt.com |    1 |          0 | 2023-05-29 11:42:22 | 2023-05-29 11:42:22 | NULL                                                                       |
| obscured@passbolt.com |    1 |          2 | 2023-05-29 11:41:58 | 2023-05-29 11:41:58 | SMTP authentication method not allowed, check if SMTP server requires TLS. |
| obscured@passbolt.com |    1 |          0 | 2023-05-29 11:41:58 | 2023-05-29 11:41:58 | NULL                                                                       |
| obscured@passbolt.com |    1 |          1 | 2023-05-29 11:41:58 | 2023-05-29 11:41:58 | SMTP authentication method not allowed, check if SMTP server requires TLS. |
| obscured@passbolt.com |    1 |          0 | 2023-05-29 11:41:58 | 2023-05-29 11:41:58 | NULL                                                                       |
| obscured@passbolt.com |    0 |          4 | 2023-05-29 11:41:49 | 2023-05-29 11:41:49 | SMTP authentication method not allowed, check if SMTP server requires TLS. |
| obscured@passbolt.com |    0 |          4 | 2023-05-29 11:12:09 | 2023-05-29 11:12:09 | SMTP authentication method not allowed, check if SMTP server requires TLS. |
| obscured@passbolt.com |    0 |          4 | 2023-05-29 11:08:24 | 2023-05-29 11:08:24 | SMTP authentication method not allowed, check if SMTP server requires TLS. |
| obscured@passbolt.com |    1 |          2 | 2023-05-29 09:38:56 | 2023-05-29 09:38:56 | SMTP authentication method not allowed, check if SMTP server requires TLS. |
| obscured@passbolt.com |    1 |          0 | 2023-05-29 09:38:56 | 2023-05-29 09:38:56 | NULL                                                                       |
| obscured@passbolt.com |    1 |          1 | 2023-05-29 09:38:56 | 2023-05-29 09:38:56 | SMTP authentication method not allowed, check if SMTP server requires TLS. |
| obscured@passbolt.com |    1 |          0 | 2023-05-29 09:38:56 | 2023-05-29 09:38:56 | NULL                                                                       |
| obscured@passbolt.com |    0 |          4 | 2023-05-29 09:38:47 | 2023-05-29 09:38:47 | SMTP authentication method not allowed, check if SMTP server requires TLS. |
| obscured@passbolt.com |    1 |          0 | 2023-05-29 08:54:40 | 2023-05-29 08:54:40 | NULL                                                                       |
| obscured@passbolt.com |    0 |          4 | 2023-05-29 08:54:40 | 2023-05-29 08:54:40 | SMTP authentication method not allowed, check if SMTP server requires TLS. |
| obscured@passbolt.com |    1 |          0 | 2023-05-29 08:54:40 | 2023-05-29 08:54:40 | NULL                                                                       |
| obscured@passbolt.com |    1 |          1 | 2023-05-29 08:54:40 | 2023-05-29 08:54:40 | SMTP authentication method not allowed, check if SMTP server requires TLS. |
| obscured@passbolt.com |    1 |          0 | 2023-05-29 08:54:23 | 2023-05-29 08:54:23 | NULL                                                                       |
| obscured@passbolt.com |    1 |          0 | 2023-05-29 08:53:56 | 2023-05-29 08:53:56 | NULL                                                                       |
| obscured@passbolt.com |    1 |          3 | 2023-05-29 08:53:56 | 2023-05-29 08:53:56 | SMTP authentication method not allowed, check if SMTP server requires TLS. |
| obscured@passbolt.com |    1 |          0 | 2023-05-29 08:53:56 | 2023-05-29 08:53:56 | NULL                                                                       |
| obscured@passbolt.com |    1 |          1 | 2023-05-29 08:53:56 | 2023-05-29 08:53:56 | SMTP authentication method not allowed, check if SMTP server requires TLS. |
| obscured@passbolt.com |    1 |          0 | 2023-05-29 08:53:44 | 2023-05-29 08:53:44 | NULL                                                                       |
+----------------------------+------+------------+---------------------+---------------------+----------------------------------------------------------------------------+
40 rows in set (0.00 sec)

One added change from the server migration is that the health check that previously was showing few errors, which to be fair we already thought were false positives, are gone and the check is now clean:

[PASS] No error found. Nice one sparky!
1 Like

I don’t recall ever seeing anything like this - it’s very interesting since it occurs on a fresh/different OS as well.

Given that you ran it by the mail team once more, and since we know that mail does deliver even without SMTP to the remote mail server, I guess a known option to probably resolve this is to configure a SMTP server locally, like with postfix, that will both offer a proper SMTP auth processs and send it on for you.

Here’s a Digital Ocean tutorial for Postfix and Ubuntu22.04.

It includes some steps for Let’s Encrypt - but you could get by with an existing cert.

The behavior is exactly the same. I setup postfix and the one-two bunch combo continues :expressionless:

5 recipients for an email notification for editing shared password. Every other send works and every other fails and one does not get send through at all even after 4 tries. Exactly the same as with SMTP config.

One thing is different though and now with postfix config the SMTP error is different:

tail -f /var/log/passbolt/cron.log
Email 1032 was not sent
Email 1033 was sent
Email 1034 was not sent
Email 1035 was sent
Email 1036 was not sent
tail: /var/log/passbolt/cron.log: file truncated
Email 1032 was not sent
Email 1034 was sent
Email 1036 was not sent
tail: /var/log/passbolt/cron.log: file truncated
Email 1032 was not sent
Email 1036 was sent
tail: /var/log/passbolt/cron.log: file truncated
Email 1032 was not sent
tail: /var/log/passbolt/cron.log: file truncated

mysql> SELECT email, id, sent, send_tries, send_at, created, error FROM email_queue ORDER BY id DESC LIMIT 40;
+----------------------------+------+------+------------+---------------------+---------------------+----------------------------------------------------------------------------+
| email                      | id   | sent | send_tries | send_at             | created             | error                                                                      |
+----------------------------+------+------+------------+---------------------+---------------------+----------------------------------------------------------------------------+
| user1@passbolt.com | 1036 |    1 |          2 | 2023-05-31 06:20:44 | 2023-05-31 06:20:44 | SMTP Error: 503 5.5.1 Error: authentication not enabled                    |
| user2@passbolt.com | 1035 |    1 |          0 | 2023-05-31 06:20:44 | 2023-05-31 06:20:44 | NULL                                                                       |
| user3@passbolt.com | 1034 |    1 |          1 | 2023-05-31 06:20:44 | 2023-05-31 06:20:44 | SMTP Error: 503 5.5.1 Error: authentication not enabled                    |
| user4@passbolt.com | 1033 |    1 |          0 | 2023-05-31 06:20:44 | 2023-05-31 06:20:44 | NULL                                                                       |
| user5@passbolt.com | 1032 |    0 |          4 | 2023-05-31 06:20:44 | 2023-05-31 06:20:44 | SMTP Error: 503 5.5.1 Error: authentication not enabled      

I confirmed from the received email that the source is now localhost / 127.0.0.1 and postfix instead of the server where Passbolt is via our SMTP server.

Received: from passboltserver@passbolt.com (localhost [127.0.0.1])
	by passboltserver@passbolt.com (Postfix) with ESMTP id 36FDF16072C
	for <myemail@ourdomain.com>; Wed, 31 May 2023 06:21:02 +0000 (UTC)
From: Passbolt <passbolt@passbolt.com>

vs

Received: from passboltserver@passbolt.com (x.x.x.x) by
 smtp.server.dns (x.x.y.y) with Microsoft SMTP Server id
 15.2.1118.26 via Frontend Transport; Wed, 31 May 2023 14:07:03 +0800
From: Passbolt <passbolt@passbolt.com>

I also did some more testing. If I do the registration in quick succession multiple times one of the emails comes through pretty reliably. Never the first one though, always second or third.

Postfix should receive the mail, and if it cannot be delivered, I believe it should be retrying them in a queue, and the ESMTP is that queue identifier. The expectation in this setup is that passbolt will always be able to hand off the message to postfix and postfix can do the retries.

There are SMTP errors in the db and you should establish authentication with postfix for passbolt to connect to. It seems this has not been done yet?