Test email sends, but newly created users do not receive invite email

[LeRocque]

Checklist
I have read intro post
I have read the tutorials, help and searched for similar issues
I provide relevant information about my server (component names and versions, etc.)
I provide a copy of my logs and healthcheck
I describe the steps I have taken to trouble shoot the problem
I describe the steps on how to reproduce the issue

System info:
– Server operating system name and version => Running in AWS EKS cluster on Debian GNU/Linux 12 (bookworm)
– Web server name and version => nginx/1.22.1
– Database server name and version => PostgreSQL 16.0 on x86_64-pc-linux-gnu
– Php version => PHP 8.2.18
– Passbolt version => 4.7.0

Healthcheck:

• Healthcheck shell

---

Environment

[PASS] PHP version 8.2.18.
[PASS] PHP version is 8.1 or above.
[PASS] PCRE compiled with unicode support.
[PASS] Mbstring extension is installed.
[PASS] Intl extension is installed.
[PASS] GD or Imagick extension is installed.
[PASS] The temporary directory and its content are writable and not executable.
[PASS] The logs directory and its content are writable.

Config files

[PASS] The application config file is present
[WARN] The passbolt config file is missing in /etc/passbolt/
[HELP] Copy /etc/passbolt/passbolt.default.php to /etc/passbolt/passbolt.php
[HELP] The passbolt config file is not required if passbolt is configured with environment variables

Core config

[PASS] Cache is working.
[PASS] Debug mode is off.
[PASS] Unique value set for security.salt
[PASS] Full base url is set to https://*****
[PASS] App.fullBaseUrl validation OK.
[PASS] /healthcheck/status is reachable.

SSL Certificate

[PASS] SSL peer certificate validates.
[PASS] Hostname is matching in SSL certificate.
[PASS] Not using a self-signed certificate.

SMTP settings

[PASS] The SMTP Settings plugin is enabled.
[PASS] SMTP Settings coherent. You may send a test email to validate them.
[WARN] The SMTP Settings source is: env variables.
[HELP] It is recommended to set the SMTP Settings in the database through the administration section.
[WARN] The SMTP Settings plugin endpoints are enabled.
[HELP] It is recommended to disable the plugin endpoints.
[HELP] Set the PASSBOLT_SECURITY_SMTP_SETTINGS_ENDPOINTS_DISABLED environment variable to true.
[HELP] Or set passbolt.security.smtpSettings.endpointsDisabled to true in /etc/passbolt/passbolt.php.

JWT Authentication

[PASS] The JWT Authentication plugin is enabled.
[PASS] The /etc/passbolt/jwt/ directory is not writable.
[PASS] A valid JWT key pair was found.

GPG Configuration

[PASS] PHP GPG Module is installed and loaded.
[PASS] The environment variable GNUPGHOME is set to /var/lib/passbolt/.gnupg.
[PASS] The directory /var/lib/passbolt/.gnupg containing the keyring is writable by the webserver user.
[PASS] The server OpenPGP key is not the default one.
[PASS] The public key file is defined in /etc/passbolt/passbolt.php and readable.
[PASS] The private key file is defined in /etc/passbolt/passbolt.php and readable.
[PASS] The server key fingerprint matches the one defined in /etc/passbolt/passbolt.php.
[PASS] The server public key defined in the /etc/passbolt/passbolt.php (or environment variables) is in the keyring.
[PASS] There is a valid email id defined for the server key.
[PASS] The public key can be used to encrypt a message.
[PASS] The private key can be used to sign a message.
[PASS] The public and private keys can be used to encrypt and sign a message.
[PASS] The private key can be used to decrypt a message.
[PASS] The private key can be used to decrypt and verify a message.
[PASS] The public key can be used to verify a signature.
[PASS] The server public key format is Gopengpg compatible.
[PASS] The server private key format is Gopengpg compatible.

Application configuration

[PASS] Using latest passbolt version (4.7.0).
[PASS] Passbolt is configured to force SSL use.
[PASS] App.fullBaseUrl is set to HTTPS.
[PASS] Selenium API endpoints are disabled.
[PASS] Search engine robots are told not to index content.
[INFO] The Self Registration plugin is enabled.
[INFO] Registration is closed, only administrators can add users.
[PASS] The deprecated self registration public setting was not found in /etc/passbolt/passbolt.php.
[WARN] Host availability checking is disabled.
[HELP] Make sure this instance is not publicly available on the internet.
[HELP] Or set the PASSBOLT_EMAIL_VALIDATE_MX environment variable to true.
[HELP] Or set passbolt.email.validate.mx to true in /etc/passbolt/passbolt.php.
[PASS] Serving the compiled version of the javascript app.
[WARN] Some email notifications are disabled by the administrator.
[PASS] The database schema up to date.

Database

[PASS] The application is able to connect to the database
[PASS] 31 tables found.
[PASS] Some default content is present.

[PASS] No error found. Nice one sparky!

Test Email:

• Email configuration

---

Host: email-smtp.us-east-2.amazonaws.com-
Port: 587
Username: ********
Password: *********
TLS: true
Sending email from: Passbolt <*********>
Sending email to: youremail@domain.com-

Trace
[220] email-smtp.amazonaws.com- ESMTP SimpleEmailService-d-NJ7WNEPV4 vESRflbfvpREl27F78p4
EHLO localhost
[250] email-smtp.amazonaws.com-
[250] 8BITMIME
[250] STARTTLS
[250] AUTH PLAIN LOGIN
[250] Ok
STARTTLS
[220] Ready to start TLS
EHLO localhost
[250] email-smtp.amazonaws.com-
[250] 8BITMIME
[250] STARTTLS
[250] AUTH PLAIN LOGIN
[250] Ok
AUTH PLAIN *****
[235] Authentication successful.
MAIL FROM:<**>
[250] Ok
RCPT TO:<youremail@domain.com->
[250] Ok
DATA
[354] End data with .
From: Passbolt <>
To: youremail@domain.com-
Date: Fri, 17 May 2024 13:17:45 +0000
Message-ID: <d0371044930d42638279fdb37c8c4824@passbolt-depl-srv-59458b9858-bbhcq->
Subject: Passbolt test email
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Congratulations!
If you receive this email, it means that your passbolt smtp configuration is working fine.

.
[250] Ok 010f018f86b38ed2-2f274272-702d-4253-9a97-008a1b00cba5-000000
QUIT
The message has been successfully sent!

Datacheck:

---

Data check shell
[PASS] Data integrity for AuthenticationTokens.
[PASS] Can validate: 3/3
[PASS] Data integrity for Comments.
[PASS] Can validate: 0/0
[PASS] Data integrity for Favorites.
[PASS] Can validate: 0/0
[PASS] Data integrity for Gpgkeys.
[PASS] Can encrypt: 1/1
[PASS] Pass validation service checks: 1/1
[PASS] Entity data and armored key data matches: 1/1
[PASS] Is not expired: 1/1
[PASS] Is armored key format valid: 1/1
[PASS] Data integrity for Groups.
[PASS] Can validate: 0/0
[PASS] Data integrity for Profiles.
[PASS] Can validate: 2/2
[PASS] Data integrity for Resources.
[PASS] Can validate: 0/0
[PASS] Data integrity for Secrets.
[PASS] Can validate: 0/0
[PASS] Data integrity for Users.
[PASS] Can validate: 2/2

I have tried to find logs for the cron job, but have not been successful. I can confirm that the cron job pods are being created and execute with no errors every minute.

I can see the two emails I expect to be sent in the database queue, but they will not send.

[LeRocque]

Can also confirm that email notifications are enabled

[pabloelcolombiano]

Hi @LeRocque ,

according to the healthcheck, you have the SMTP settings defined in env variable.

@LeRocque

[WARN] The SMTP Settings source is: env variables.

Although this is also supported, maybe you could try setting the SMTP settings under the Email server section.

Also you said you accessed to the email_queue in DB. Do you see any error messages in the error column?

select id, email, template, error,sent, locked, send_tries from email_queue\G

[LeRocque]

I have tried setting the SMTP settings under the Email server section and I get the same result. The test email sends (to anyone), but user registration emails never go out.

There are no error messages in the error column

[pabloelcolombiano]

Thank you for the info.

How about in the folowing columns: template, error, sent, locked and send_tries ?

[LeRocque]

[pabloelcolombiano]

Thank you @LeRocque , with this info we can investigate a bit deeper.

What happens if you set the locked field to false manually? For some reason I cannot explain, this field is set to false on your instance.

As a temporary solution, you can also register users via the command, which will generate a link that you can provide to the user.
bin/cake passbolt register_user -i

[LeRocque]

Manually setting locked to false does work, but they revert back to locked again after the next cron job runs and never send.

I am able to use the temporary solution by registering users with the command, but I am testing this out for my company so I need the emails to work properly.

[LeRocque]

I decided to use a free trail of the Passbolt Pro Edition to see if that would fix the issue, but nothing has changed.

[clayton]

@LeRocque

– Server operating system name and version => Running in AWS EKS cluster on Debian GNU/Linux 12 (bookworm)

You are using the helm chart, right?

If so can you take a look at the pods? There should be one that is handling the email cronjob and I’ve seen it before where some cluster configurations make it where that pod isn’t able to reach any others and therefore it never tries to send the emails

[LeRocque]

Hello clayton, the cron job pod gets created every minute and executes with no errors. The logs in the cron job simply say “Sending Emails” and then the pod terminates.

Also worth noting here that I have disabled Redis at this point too, just to take that out of possible causes of this email problem.

Here are some screenshots from K9s.

[LeRocque]

This was resolved thanks to Anthony via support email.

I added an env variable of EMAIL_DEFAULT_TRANSPORT: AWS SES to my values.yaml file.

This created some sort of bug that overwrote the default transport method of the cron job, but did not throw any errors and still allowed the SMTP test to pass.