Sending email by CLI works just fine, but cronjob do not

Installation Issues
1

Checklist
I have read intro post: About the Installation Issues category
I have read the tutorials, help and searched for similar issues
I provide relevant information about my server (component names and versions, etc.)
I provide a copy of my logs and healthcheck
I describe the steps I have taken to trouble shoot the problem
I describe the steps on how to reproduce the issue

Hello, guys. I’m migrating your passbolt from an kubernetes cluster to another and something really strage is heppening.

My notification emails are geting queued and after some minutes they all got locked. They show “1” on the “locked” colunm at the table “email_queue”.

Strange is that when I run the cron command to clear the locks at the cli:

su -s /bin/bash -c “/usr/share/php/passbolt/bin/cake sender clear_locks” www-data

And then exec:

su -c “$PASSBOLT_BASE_DIR/bin/cake EmailQueue.sender >> /usr/share/php/passbolt/cron.log” -s /bin/bash www-data

The email are sent normally.

My healthcheck:

 ____                  __          ____  
/ __ \____  _____ ____/ /_  ____  / / /_

/ // / __ `/ / / __ / __ / / _/
/ / // ( | ) // / // / / /
// _,///./_//__/

Open source password manager for teams

Healthcheck shell

Environment

[PASS] PHP version 7.3.29-1~deb10u1.
[PASS] PCRE compiled with unicode support.
[PASS] The temporary directory and its content are writable.
[PASS] The public image directory and its content are writable.
[PASS] The logs directory and its content are writable.
[PASS] GD or Imagick extension is installed.
[PASS] Intl extension is installed.
[PASS] Mbstring extension is installed.

Config files

[PASS] The application config file is present
[WARN] The passbolt config file is missing in /etc/passbolt/
[HELP] Copy /etc/passbolt/passbolt.default.php to /etc/passbolt/passbolt.php
[HELP] The passbolt config file is not required if passbolt is configured with environment variables

Core config

[FAIL] Debug mode is on.
[HELP] Set debug = false; in config/passbolt.php
[PASS] Cache is working.
[PASS] Unique value set for security.salt
[PASS] Full base url is set to https://...
[PASS] App.fullBaseUrl validation OK.
[PASS] /healthcheck/status is reachable.

SSL Certificate

[FAIL] SSL peer certificate does not validate
[FAIL] Hostname does not match when validating certificates.
[WARN] Using a self-signed certificate
[HELP] fopen(): SSL operation failed with code 1. OpenSSL Error messages:
error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed
fopen(): Failed to enable crypto
fopen(https://.../healthcheck/status.json): failed to open stream: operation failed

Database

[PASS] The application is able to connect to the database
[PASS] 25 tables found
[PASS] Some default content is present
[PASS] The database schema up to date.

GPG Configuration

[PASS] PHP GPG Module is installed and loaded.
[PASS] The environment variable GNUPGHOME is set to /var/lib/passbolt/.gnupg.
[PASS] The directory /var/lib/passbolt/.gnupg containing the keyring is writable by the webserver user.
[PASS] The server OpenPGP key is not the default one
[PASS] The public key file is defined in config/passbolt.php and readable.
[PASS] The private key file is defined in config/passbolt.php and readable.
[PASS] The server key fingerprint matches the one defined in config/passbolt.php.
[PASS] The server public key defined in the config/passbolt.php (or environment variables) is in the keyring.
[PASS] There is a valid email id defined for the server key.
[PASS] The public key can be used to encrypt a message.
[PASS] The private key can be used to sign a message.
[PASS] The public and private keys can be used to encrypt and sign a message.
[PASS] The private key can be used to decrypt a message.
[PASS] The private key can be used to decrypt and verify a message.
[PASS] The public key can be used to verify a signature.

Application configuration

[PASS] Using latest passbolt version (3.2.1).
[PASS] Passbolt is configured to force SSL use.
[PASS] App.fullBaseUrl is set to HTTPS.
[PASS] Selenium API endpoints are disabled.
[PASS] Search engine robots are told not to index content.
[PASS] Registration is closed, only administrators can add users.
[PASS] Serving the compiled version of the javascript app
[PASS] All email notifications will be sent.

[FAIL] 3 error(s) found. Hang in there!

My env from helm chart:

PASSBOLT_EMAIL_SEND_PASSWORD_CREATE: true
PASSBOLT_EMAIL_SEND_PASSWORD_UPDATE: true
PASSBOLT_EMAIL_SHOW_SECRET: false
PASSBOLT_EMAIL_SHOW_URI: false
PASSBOLT_EMAIL_SHOW_USERNAME: false
DEBUG: true
EMAIL_DEFAULT_FROM: noreply@..*
EMAIL_TRANSPORT_DEFAULT_CLASS_NAME: Smtp
EMAIL_TRANSPORT_DEFAULT_HOST: ..
EMAIL_TRANSPORT_DEFAULT_PORT: 587
PASSBOLT_EMAIL_SEND_USER_CREATE: true
PASSBOLT_EMAIL_SEND_USER_RECOVER: true

The test email is working fine either:

 ____                  __          ____  
/ __ \____  _____ ____/ /_  ____  / / /_

/ // / __ `/ / / __ / __ / / _/
/ / // ( | ) // / // / / /
// _,///./_//__/

Open source password manager for teams

Debug email shell

Email configuration

Host: ..*****
Port: 587
Username: *****
Password: *********
TLS: false
Sending email from: Passbolt <noreply@.edu.br>
Sending email to: ****.@*****.edu.br

Trace
[220] saasauth0007.correio.pw ESMTP Postfix (Debian/GNU)

EHLO localhost
[250] ..*****
[250] PIPELINING
[250] SIZE 37527614
[250] VRFY
[250] ETRN
[250] STARTTLS
[250] AUTH PLAIN LOGIN
[250] ENHANCEDSTATUSCODES
[250] 8BITMIME
[250] DSN
AUTH PLAIN AHVuaWZhY2lzYQB3RW5KZVlxbDgzOTQ=
[235] 2.7.0 Authentication successful
MAIL FROM:<@.edu.br>
[250] 2.1.0 Ok
RCPT TO:<.@.edu.br>
[250] 2.1.5 Ok
DATA
[354] End data with .
From: Passbolt <@.edu.br>
To: .@.edu.br
Date: Sat, 31 Jul 2021 23:34:51 +0000
Message-ID: 012aebe3cb434d70a457343e70690272@passbolt-677f684c9f-tgchc
Subject: Passbolt test email
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Congratulations!
If you receive this email, it means that your passbolt smtp configuration is working fine.

.
[250] 2.0.0 Ok: queued as 4F5E4180ACD

QUIT
The message has been successfully sent!

2

Hi @hpribeiro

Just to confirm, did you receive the actual email?

Are you wanting STARTTLS?

I see STARTTLS and TLS in the config is false by default.
EMAIL_TRANSPORT_DEFAULT_TLS=true to enable.

If you don’t want STARTTLS see this PR on github. Docker/cakephp has trouble with this particular variable.

3

Hello @garrett

I receive the actual email only if I clear the locks then run email_queue.sender at the CLI.

su -s /bin/bash -c “/usr/share/php/passbolt/bin/cake sender clear_locks” www-data
su -s /bin/bash -c “/usr/share/php/passbolt/bin/cake sender” www-data

Email 1 was sent

If I don’t do that procedure, the email stays enqueued forever locked.

And I do not want to use TLS. I tried to force false like you said, by editing the config file of passbolt:

‘tls’ => filter_var(env(‘EMAIL_TRANSPORT_DEFAULT_TLS’, false), FILTER_VALIDATE_BOOLEAN),

But still not working.

5

Similar, start here in this thread. Does it help?

6

Its similar, but in my case send_tries don’t even increases, it’s always 0.

Isn’t this strange, since executing email_queue.sender I receive the email, but running from cronjob the email just get locked?

I’m stucked on this for 4 days now, and running out of options. I read almost every thread here about email issues. :frowning:

7

@max What do you think about this?

8

Hi @hpribeiro (and @garrett),

Can you confirmed that you have similar output:

root@passbolt-pod-xxx:/# cat /etc/cron.d/passbolt_email
* * * * * su -c "source /etc/environment ; /var/www/passbolt/bin/cake EmailQueue.sender" -s /bin/bash www-data >> /var/log/cron.log 2>&1

And if so, can you have a look at the content of your log file: /var/log/cron.log

Thanks,
Max

9

Hello @max,

First I want to thank you and @garrett a big time for trying to help me.

As I’m using the latest docker image my cron looks like this:

root@passbolt-68dff48684-jf5zt:/usr/share/php/passbolt# cat /etc/cron.d/passbolt-ce-server

PATH=/bin:/usr/local/bin:/usr/bin
PASSBOLT_BASE_DIR=/usr/share/php/passbolt

* * * * * www-data exec /bin/bash -c ". /etc/environment && $PASSBOLT_BASE_DIR/bin/cron"

And about the logs, my cron service is managed by supervisord and sending its logs to stderr:

root@passbolt-68dff48684-jf5zt:/usr/share/php/passbolt# cat /etc/supervisor/conf.d/cron.conf 
[program:cron]
command=/bin/bash -c "declare -p | grep -Ev 'BASHOPTS|BASH_VERSINFO|EUID|PPID|SHELLOPTS|UID' > /etc/environment; cron -f -l"
autostart=true
priority=20
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0

Is there a way to watch cron logs in another way?

10

@hpribeiro docker-compose logs | Docker Documentation I would recommend writing out to a file docker-compose logs passbolt > logfile.

You could also run the container without being in detached mode docker-compose up instead of docker-compose up -d. This is regarding the use of Docker itself at this point.

If there is an error you will see it in the logs. If it’s an error at startup when supervisor is attempting to run the service it may give up.

You could also check to see if the cron service is running. ps aux from within the container or via docker-compose exec command from outside.

11

Thank you @garrett!

I found no errors in my logs container logs other than that:

INFO reaped unknown pid 370453

Another thing has come to my attention, is the timezone have something to do with email queue? The container is +3h advanced in time from my actual timezone.

12

@hpribeiro I think we need to keep looking at logs for errors. At long as the time is correct for the given timezone I would think it should be okay.

Please post results of docker-compose exec passbolt ps aux