Unable to get mails for Account Creation, Test Mails work OK

re_moto · February 19, 2024, 8:46pm

Checklist
I have read intro post
I have read the tutorials, help and searched for similar issues
I provide relevant information about my server (component names and versions, etc.)
I provide a copy of my logs and healthcheck
I describe the steps I have taken to trouble shoot the problem
I describe the steps on how to reproduce the issue

After setting up Passbolt Community edition via Helm chart using my own domain name, I am able to create my user, connect to front end, add new users, administer, everything is working as expected. I have configured my SMTP options in the front end (so they are in the db) and I am able to successfully receive the test mails. I also see these in SendGrid activity (I am able to see every single mail that is send via SendGrid).

However, the mails that should be sent in the background, are not operational. Specifically, when I add new users, and send/re-send activation mails, these are never sent. I tried multiple times and these mails are never sent. I see records created in MariaDB email_queue table, so these are meant by Passbolt to be sent. However, these never make it to SendGrid (I see every test mail there, but none of the scheduled ones ever show up.)

How can I investigate further why the mails are not sent in the background? Are there any logs or logging options to enable to get the output of when Passbolt is supposed to send the mails?

Details of the install:
Helm via chart chart: passbolt-0.7.2

./bin/cake passbolt healthcheck

 ____                  __          ____
/ __ \____  _____ ____/ /_  ____  / / /_

/ // / __ `/ / / __ / __ / / _/
/ / // ( | ) // / // / / /
// _,///./_//__/

Open source password manager for teams

Healthcheck shell

Environment

[PASS] PHP version 8.2.7.
[PASS] PHP version is 8.1 or above.
[PASS] PCRE compiled with unicode support.
[PASS] The temporary directory and its content are writable and not executable.
[PASS] The logs directory and its content are writable.
[PASS] GD or Imagick extension is installed.
[PASS] Intl extension is installed.
[PASS] Mbstring extension is installed.

Config files

[PASS] The application config file is present
[WARN] The passbolt config file is missing in /etc/passbolt/
[HELP] Copy /etc/passbolt/passbolt.default.php to /etc/passbolt/passbolt.php
[HELP] The passbolt config file is not required if passbolt is configured with environment variables

Core config

[PASS] Debug mode is off.
[PASS] Cache is working.
[PASS] Unique value set for security.salt
[PASS] Full base url is set to --removed–
[PASS] App.fullBaseUrl validation OK.
[PASS] /healthcheck/status is reachable.

SSL Certificate

[WARN] SSL peer certificate does not validate
[WARN] Hostname does not match when validating certificates.
[WARN] Using a self-signed certificate
[HELP] Check Troubleshoot-ssl
[HELP] cURL Error (60) SSL certificate problem: self-signed certificate

Database

[PASS] The application is able to connect to the database
[PASS] 31 tables found
[PASS] Some default content is present
[PASS] The database schema up to date.

GPG Configuration

[PASS] PHP GPG Module is installed and loaded.
[PASS] The environment variable GNUPGHOME is set to /var/lib/passbolt/.gnupg.
[PASS] The directory /var/lib/passbolt/.gnupg containing the keyring is writable by the webserver user.
[PASS] The server OpenPGP key is not the default one
[PASS] The public key file is defined in /etc/passbolt/passbolt.php and readable.
[PASS] The private key file is defined in /etc/passbolt/passbolt.php and readable.
[PASS] The server key fingerprint matches the one defined in /etc/passbolt/passbolt.php.
[PASS] The server public key defined in the /etc/passbolt/passbolt.php (or environment variables) is in the keyring.
[PASS] There is a valid email id defined for the server key.
[PASS] The public key can be used to encrypt a message.
[PASS] The private key can be used to sign a message.
[PASS] The public and private keys can be used to encrypt and sign a message.
[PASS] The private key can be used to decrypt a message.
[PASS] The private key can be used to decrypt and verify a message.
[PASS] The public key can be used to verify a signature.
[PASS] The server public key format is Gopengpg compatible.
[PASS] The server private key format is Gopengpg compatible.

Application configuration

[PASS] Using latest passbolt version (4.5.2).
[PASS] Passbolt is configured to force SSL use.
[PASS] App.fullBaseUrl is set to HTTPS.
[PASS] Selenium API endpoints are disabled.
[PASS] Search engine robots are told not to index content.
[INFO] The Self Registration plugin is enabled.
[INFO] Registration is closed, only administrators can add users.
[PASS] The deprecated self registration public setting was not found in /etc/passbolt/passbolt.php.
[WARN] Host availability checking is disabled.
[HELP] Make sure this instance is not publicly available on the internet.
[HELP] Or set the PASSBOLT_EMAIL_VALIDATE_MX environment variable to true.
[HELP] Or set passbolt.email.validate.mx to true in /etc/passbolt/passbolt.php.
[PASS] Serving the compiled version of the javascript app.
[WARN] Some email notifications are disabled by the administrator.

JWT Authentication

[PASS] The JWT Authentication plugin is enabled
[FAIL] The /etc/passbolt/jwt/ directory should not be writable.
[HELP] You can try:
[HELP] sudo chown -Rf root:www-data /etc/passbolt/jwt/
[HELP] sudo chmod 750 /etc/passbolt/jwt/
[HELP] sudo chmod 640 /etc/passbolt/jwt/jwt.key
[HELP] sudo chmod 640 /etc/passbolt/jwt/jwt.pem
[PASS] A valid JWT key pair was found

SMTP Settings

[PASS] The SMTP Settings plugin is enabled.
[PASS] SMTP Settings coherent. You may send a test email to validate them.
[PASS] The SMTP Settings source is: database.
[WARN] The SMTP Settings plugin endpoints are enabled.
[HELP] It is recommended to disable the plugin endpoints.
[HELP] Set the PASSBOLT_SECURITY_SMTP_SETTINGS_ENDPOINTS_DISABLED environment variable to true.
[HELP] Or set passbolt.security.smtpSettings.endpointsDisabled to true in /etc/passbolt/passbolt.php.

[FAIL] 1 error(s) found. Hang in there!

re_moto · February 19, 2024, 8:48pm

As for reproducing the issue, I install via the Helm chart on AKS. I enable the ingress to be able to access the sever with my DNS. I don’t think I have customized anything that should affect the e-mail being sent. I setup my SMTP setting for SendGrid and have no problem with the test mails, they work 100% of the time.

clayton · February 20, 2024, 7:00am

hey @re_moto welcome to the forum!

Since those are showing as not even attempted to send you’ll want to check the cronjob. For the helm chart you should see a pod for this, could you check to see if there are any issues with that?

re_moto · February 20, 2024, 9:26pm

Hi @Clayton, thanks for the welcome!

Indeed I see the cronjob running and there are warnings, but nothing reported after “Sending mails”

Cronjobs:
┌────────────────────────────────────────Cronjobs(passbolt)[1]
│ NAME↑ SCHEDULE SUSPEND ACTIVE LAST_SCHEDULE AGE
│ my-passbolt-cron-proc-email * * * * * false 0 56s 5d19h

Jobs:
┌──── Jobs(passbolt/my-passbolt-cron-proc-email)[3] ───┐
│ NAME COMPLETIONS DURATION AGE↑ │
│ my-passbolt-cron-proc-email-28474331 1/1 19s 56s │
│ my-passbolt-cron-proc-email-28474330 1/1 19s 116s │
│ my-passbolt-cron-proc-email-28474329 1/1 18s 2m56s |

Logs from the latest Job:
│ 2024-02-20T21:23:03.241514953Z my-passbolt-cron-proc-email-28474403-lvr76 my-passbolt-cron-proc-email-redisproxy [WARNING] (7) : Server bk_redis_master/R0 is DOWN, reason: Layer7 timeout, │
│ 2024-02-20T21:23:03.241592354Z my-passbolt-cron-proc-email-28474403-lvr76 my-passbolt-cron-proc-email-redisproxy Server bk_redis_master/R0 is DOWN, reason: Layer7 timeout, info: " at step 6 │
│ 2024-02-20T21:23:03.564464581Z my-passbolt-cron-proc-email-28474403-lvr76 my-passbolt-cron-proc-email-redisproxy [WARNING] (7) : Server bk_redis_master/R1 is DOWN, reason: Layer7 timeout, │
│ 2024-02-20T21:23:03.564511782Z my-passbolt-cron-proc-email-28474403-lvr76 my-passbolt-cron-proc-email-redisproxy Server bk_redis_master/R1 is DOWN, reason: Layer7 timeout, info: " at step 6 │
│ 2024-02-20T21:23:10.787553774Z my-passbolt-cron-proc-email-28474403-lvr76 my-passbolt-cron-proc-email Sending emails │
│ 2024-02-20T21:23:51.085697Z Stream closed EOF for passbolt/my-passbolt-cron-proc-email-28474403-lvr76 (my-passbolt-cron-proc-email-redisproxy) │
│ 2024-02-20T21:23:51.085697Z Stream closed EOF for passbolt/my-passbolt-cron-proc-email-28474403-lvr76 (my-passbolt-cron-proc-email)

re_moto · February 21, 2024, 5:25pm

FYI, I wanted to compare an installation with working emails so I also installed Passbolt on Ubuntu 22.04 by following these steps:

The installation went without any problems and I configured SMTP identically to my Helm install. On this Ubuntu 22.04 install, all the mails (test and background mails) work without any problem. That probably helps establish that SMTP settings / SendGrid mail service are not the source of the issue.

I also noticed that in the Helm installation this is the argument that triggers mail sending:

spec:
containers:

args:
- -c
- |
  sleep 10
  echo “Sending emails”
  && bin/cron
  && touch /tmp/pod/succes

Running bin/cron directly in the deployment container produces no output. How can I dig further into the activity that needs to happen when bin/cron? Assuming this is the command that triggers the mail sending.

clayton · February 26, 2024, 7:25am

Good call on testing the credentials elsewhere!

This would seem like an issue with the pods communicating with each other then. Could you run a describe on the cronjob deployment to see if anything is going on there?

I have seen before where everything seemed alright but due to some annotations being missing in a customized environment that the pods couldn’t reach each other