Passbolt broken after v4 update - intermittent login issues

API & Customization issues
1

Hello dears,

We’ve been using passbolt for more than 2 years now.
After upgrading from v3.12.0 to v4.0.2 the API crashes after some time, and is very inconsistent.

It throws some of our users out and they cannot login (even when they start a account recovery).

It will sometimes show this error, even though the user exists and restarting the docker compose services will fix this (a restart does not always fix it, sometimes we just have to wait it out XD)

image
image492×697 24.9 KB

We do not have any idea what caused this issue after the upgrade.

 docker compose exec -ti passbolt cat /etc/passbolt/version.php

<?php
return [
    'passbolt' => [
        'version' => '4.0.2',
        'name' => 'Creep',
    ]
];
2

Is there anything in the stdout logs that you can share with us?

3

Hello diego,

We’ve been looking at the logs and we cannot find anything related. all seems ok (there was one 500 error once in the past week and we cannot find it in the logs anymore)

the issue is, throughout the day. different users are logged out and it seems like the system forgets they were ever a user as it asks them to provide their private key, one we do provide it it says key does not match any account.

That same user, might wait for a few hours or a passbolt restart, and they can log in with no issues.

 docker compose exec passbolt /usr/share/php/passbolt/bin/status-report

     ____                  __          ____
    / __ \____  _____ ____/ /_  ____  / / /_
   / /_/ / __ `/ ___/ ___/ __ \/ __ \/ / __/
  / ____/ /_/ (__  |__  ) /_/ / /_/ / / /
 /_/    \__,_/____/____/_.___/\____/_/\__/

 Open source password manager for teams
-------------------------------------------------------------------------------
Passbolt CE 4.0.2
Cakephp 4.4.11
Linux 4d6333bd37cb 5.4.0-152-generic #169-Ubuntu SMP Tue Jun 6 22:23:09 UTC 2023 x86_64 GNU/Linux
PHP 8.2.7 (cli) (built: Jun  9 2023 19:37:27) (NTS)
mysql  Ver 15.1 Distrib 10.11.3-MariaDB, for debian-linux-gnu (x86_64) using  EditLine wrapper
gpg: WARNING: unsafe ownership on homedir '/var/lib/passbolt/.gnupg'
gpg (GnuPG) 2.2.40
 ERROR: /usr/share/php/passbolt/bin/utils.sh: line 64: composer: command not found

     ____                  __          ____
    / __ \____  _____ ____/ /_  ____  / / /_
   / /_/ / __ `/ ___/ ___/ __ \/ __ \/ / __/
  / ____/ /_/ (__  |__  ) /_/ / /_/ / / /
 /_/    \__,_/____/____/_.___/\____/_/\__/

 Open source password manager for teams
-------------------------------------------------------------------------------
 Healthcheck shell
-------------------------------------------------------------------------------

 Environment

 [PASS] PHP version 8.2.7.
 [PASS] PCRE compiled with unicode support.
 [PASS] The temporary directory and its content are writable and not executable.
 [PASS] The logs directory and its content are writable.
 [PASS] GD or Imagick extension is installed.
 [PASS] Intl extension is installed.
 [PASS] Mbstring extension is installed.

 Config files

 [PASS] The application config file is present
 [WARN] The passbolt config file is missing in /etc/passbolt/
 [HELP] Copy /etc/passbolt/passbolt.default.php to /etc/passbolt/passbolt.php
 [HELP] The passbolt config file is not required if passbolt is configured with environment variables

 Core config

 [PASS] Debug mode is off.
 [PASS] Cache is working.
 [PASS] Unique value set for security.salt
 [PASS] Full base url is set to https://passbolt.dev.krd
 [PASS] App.fullBaseUrl validation OK.
 [PASS] /healthcheck/status is reachable.

 SSL Certificate

 [PASS] SSL peer certificate validates
 [PASS] Hostname is matching in SSL certificate.
 [PASS] Not using a self-signed certificate

 Database

 [PASS] The application is able to connect to the database
 [PASS] 30 tables found
 [PASS] Some default content is present
 [PASS] The database schema up to date.

 GPG Configuration

 [PASS] PHP GPG Module is installed and loaded.
 [PASS] The environment variable GNUPGHOME is set to /var/lib/passbolt/.gnupg.
 [PASS] The directory /var/lib/passbolt/.gnupg containing the keyring is writable by the webserver user.
 [FAIL] The server OpenPGP key is not set
 [HELP] Create a key, export it and add the fingerprint to /etc/passbolt/passbolt.php
 [HELP] See. https://www.passbolt.com/help/tech/install#toc_gpg
 [PASS] The public key file is defined in /etc/passbolt/passbolt.php and readable.
 [PASS] The private key file is defined in /etc/passbolt/passbolt.php and readable.
 [FAIL] The server key fingerprint doesnt match the one defined in /etc/passbolt/passbolt.php.
 [HELP] Double check the key fingerprint, example:
 [HELP] sudo su -s /bin/bash -c "gpg --list-keys --fingerprint --home /var/lib/passbolt/.gnupg" www-data | grep -i -B 2 'SERVER_KEY_EMAIL'
 [HELP] SERVER_KEY_EMAIL: The email you used when you generated the server key.
 [HELP] See. https://www.passbolt.com/help/tech/install#toc_gpg
 [FAIL] The server public key defined in the /etc/passbolt/passbolt.php (or environment variables) is not in the keyring
 [HELP] Import the private server key in the keyring of the webserver user.
 [HELP] you can try:
 [HELP] sudo su -s /bin/bash -c "gpg --home /var/lib/passbolt/.gnupg --import /etc/passbolt/gpg/serverkey_private.asc" www-data
 [FAIL] The server key does not have a valid email id.
 [HELP] Edit or generate another key with a valid email id.

 Application configuration

 [PASS] Using latest passbolt version (4.0.2).
 [PASS] Passbolt is configured to force SSL use.
 [PASS] App.fullBaseUrl is set to HTTPS.
 [PASS] Selenium API endpoints are disabled.
 [PASS] Search engine robots are told not to index content.
 [INFO] The Self Registration plugin is enabled.
 [INFO] Registration is closed, only administrators can add users.
 [PASS] The deprecated self registration public setting was not found in /etc/passbolt/passbolt.php.
 [WARN] Host availability checking is disabled.
 [HELP] Make sure this instance is not publicly available on the internet.
 [HELP] Or set the PASSBOLT_EMAIL_VALIDATE_MX environment variable to true.
 [HELP] Or set passbolt.email.validate.mx to true in /etc/passbolt/passbolt.php.
 [PASS] Serving the compiled version of the javascript app.
 [WARN] Some email notifications are disabled by the administrator.

 JWT Authentication

 [PASS] The JWT Authentication plugin is enabled
 [PASS] The /etc/passbolt/jwt/ directory is not writable.
 [PASS] A valid JWT key pair was found

 SMTP Settings

 [PASS] The SMTP Settings plugin is enabled.
 [FAIL] SMTP Setting errors: App\Utility\OpenPGP\Backends\Gnupg::setDecryptKeyFromFingerprint(): Argument #1 ($fingerprint) must be of type string, null given, called in /usr/share/php/passbolt/plugins/PassboltCe/SmtpSettings/src/Service/SmtpSettingsGetSettingsInDbService.php on line 109
 [WARN] The SMTP Settings source is: undefined.
 [HELP] It is recommended to set the SMTP Settings in the database through the administration section.
 [WARN] The SMTP Settings plugin endpoints are enabled.
 [HELP] It is recommended to disable the plugin endpoints.
 [HELP] Set the PASSBOLT_SECURITY_SMTP_SETTINGS_ENDPOINTS_DISABLED environment variable to true.
 [HELP] Or set passbolt.security.smtpSettings.endpointsDisabled to true in /etc/passbolt/passbolt.php.

 [FAIL] 5 error(s) found. Hang in there!


     ____                  __          ____
    / __ \____  _____ ____/ /_  ____  / / /_
   / /_/ / __ `/ ___/ ___/ __ \/ __ \/ / __/
  / ____/ /_/ (__  |__  ) /_/ / /_/ / / /
 /_/    \__,_/____/____/_.___/\____/_/\__/

 Open source password manager for teams
-------------------------------------------------------------------------------
 Cleanup shell (dry-run)
-------------------------------------------------------------------------------
No issue found, data looks squeaky clean!

     ____                  __          ____
    / __ \____  _____ ____/ /_  ____  / / /_
   / /_/ / __ `/ ___/ ___/ __ \/ __ \/ / __/
  / ____/ /_/ (__  |__  ) /_/ / /_/ / / /
 /_/    \__,_/____/____/_.___/\____/_/\__/

 Open source password manager for teams
-------------------------------------------------------------------------------
Data check shell
[PASS] Data integrity for AuthenticationTokens.
  [PASS] Can validate: 5065/5065
[PASS] Data integrity for Comments.
  [PASS] Can validate: 0/0
[PASS] Data integrity for Favorites.
  [PASS] Can validate: 7/7
[FAIL] Data integrity for Gpgkeys.
  [FAIL] Can encrypt: 27/33
    [FAIL] Failed to encrypt with key 7BA18F137CAF2AD47602F5D70D1465FF8E54F8C4Q. Could not import the user OpenPGP key.
    [FAIL] Failed to encrypt with key E58599258FC30BD443918E7A75272900CA7E0DE3Q. Could not import the user OpenPGP key.
    [FAIL] Failed to encrypt with key 9ED4182C336A03DCBB3928C1F4A098D575F13C2CQ. Could not import the user OpenPGP key.
    [FAIL] Failed to encrypt with key 1EB47508C31498F0B580438D671BF639FFEA1821Q. Could not import the user OpenPGP key.
    [FAIL] Failed to encrypt with key 4B9ADB17A61B5824CB03E69233C654DE7BEF8349Q. Could not import the user OpenPGP key.
    [FAIL] Failed to encrypt with key BDF361329C1F591B4FFCCD9A83FB475685D243ACQ. Could not import the user OpenPGP key.
  [PASS] Pass validation service checks: 33/33
  [PASS] Entity data and armored key data matches: 33/33
  [PASS] Is not expired: 33/33
  [PASS] Is armored key format valid: 33/33
[PASS] Data integrity for Groups.
  [PASS] Can validate: 2/2
[PASS] Data integrity for Profiles.
  [PASS] Can validate: 48/48
[PASS] Data integrity for Resources.
  [PASS] Can validate: 1073/1073
[PASS] Data integrity for Secrets.
  [PASS] Can validate: 1018/1018
[PASS] Data integrity for Users.
  [PASS] Can validate: 48/48
1 Like
4

Have you tried to solve the [FAIL] items on the healthcheck command?
There are some errors that need some review on your config and maybe cause this unusual behaviour

5

gpg: WARNING: unsafe ownership on homedir ‘/var/lib/passbolt/.gnupg’

Too wide permissions on the folder holding the gnupg keyring will prevent it to work. I would look into that first.

6

We’re using the docker-compose provided in the passbolt_docker git repo, Are there any resources to guide us through these failures?

7

Here is the docker-compose file:

---
version: '3.9'
services:
  db:
    image: mariadb:10.11
    restart: unless-stopped
    env_file:
      - mysql.env
    volumes:
      - database_volume:/var/lib/mysql

  passbolt:
    image: passbolt/passbolt:4.0.2-2-ce
    restart: unless-stopped
    depends_on:
      - db
    env_file:
      - passbolt.env
    volumes:
      - gpg_volume:/etc/passbolt/gpg
      - jwt_volume:/etc/passbolt/jwt
    command: ["/usr/bin/wait-for.sh", "-t", "0", "db:3306", "--", "/docker-entrypoint.sh"]

volumes:
  gpg_volume:
  jwt_volume:
  database_volume:
8

As you are not mounting the keyring folder, it appears you’ll need to enter into the container to investigate.

docker-compose exec passbolt /bin/bash

Then inspect file permissions on /var/lib/passbolt/.gnupg and check that it’s 700. If not, chmod 700 /var/lib/passbolt/.gnupg and it should take effect immediately.

9

I tried doing this but I don’t really see the point here, as this path is not mounted in the container and upon deleting the container the contents of this file will be reset anyways.

I checked the installation docs for passbolt on docker again and there is no mention of this path nor is it in the default docker-compose file.

10

The point was you asked how to do it.

It is true that it should be as described on a fresh install. Are you saying that it is not?

11

No it is not on the docker installation, and it did not fix our issue.

12

If the file permissions are not at described they should be, please share what you found, thanks.

13

So what I did was convert the docker installation to an Ubuntu installation.

Took a backup of the docker database and server keys and imported them in the machine, so we’re now completely an ubuntu based installation.

still, some of our users cannot log back into passbolt.

/usr/share/php/passbolt/bin/version


     ____                  __          ____
    / __ \____  _____ ____/ /_  ____  / / /_
   / /_/ / __ `/ ___/ ___/ __ \/ __ \/ / __/
  / ____/ /_/ (__  |__  ) /_/ / /_/ / / /
 /_/    \__,_/____/____/_.___/\____/_/\__/

 Open source password manager for teams
-------------------------------------------------------------------------------
Passbolt CE 4.0.2
Cakephp 4.4.11
Linux pwdmngr 5.4.0-152-generic #169-Ubuntu SMP Tue Jun 6 22:23:09 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
PHP 7.4.3-4ubuntu2.18 (cli) (built: Feb 23 2023 12:43:23) ( NTS )
mysql  Ver 8.0.33-0ubuntu0.20.04.2 for Linux on x86_64 ((Ubuntu))
gpg (GnuPG) 2.2.19
libgcrypt 1.8.5
 ERROR: /usr/share/php/passbolt/bin/utils.sh: line 64: composer: command not found

/usr/share/php/passbolt/bin/healthcheck

     ____                  __          ____
    / __ \____  _____ ____/ /_  ____  / / /_
   / /_/ / __ `/ ___/ ___/ __ \/ __ \/ / __/
  / ____/ /_/ (__  |__  ) /_/ / /_/ / / /
 /_/    \__,_/____/____/_.___/\____/_/\__/

 Open source password manager for teams
-------------------------------------------------------------------------------
 Healthcheck shell
-------------------------------------------------------------------------------

 Environment

 [PASS] PHP version 7.4.3-4ubuntu2.18.
 [PASS] PCRE compiled with unicode support.
 [PASS] The temporary directory and its content are writable and not executable.
 [PASS] The logs directory and its content are writable.
 [PASS] GD or Imagick extension is installed.
 [PASS] Intl extension is installed.
 [PASS] Mbstring extension is installed.

 Config files

 [PASS] The application config file is present
 [PASS] The passbolt config file is present

 Core config

 [PASS] Debug mode is off.
 [PASS] Cache is working.
 [PASS] Unique value set for security.salt
 [PASS] Full base url is set to https://passbolt.dev.krd
 [PASS] App.fullBaseUrl validation OK.
 [PASS] /healthcheck/status is reachable.

 SSL Certificate

 [PASS] SSL peer certificate validates
 [PASS] Hostname is matching in SSL certificate.
 [PASS] Not using a self-signed certificate

 Database

 [PASS] The application is able to connect to the database
 [PASS] 30 tables found
 [PASS] Some default content is present
 [PASS] The database schema up to date.

 GPG Configuration

 [PASS] PHP GPG Module is installed and loaded.
 [PASS] The environment variable GNUPGHOME is set to /var/lib/passbolt/.gnupg.
 [PASS] The directory /var/lib/passbolt/.gnupg containing the keyring is writable by the webserver user.
 [PASS] The server OpenPGP key is not the default one
 [PASS] The public key file is defined in /etc/passbolt/passbolt.php and readable.
 [PASS] The private key file is defined in /etc/passbolt/passbolt.php and readable.
 [PASS] The server key fingerprint matches the one defined in /etc/passbolt/passbolt.php.
 [PASS] The server public key defined in the /etc/passbolt/passbolt.php (or environment variables) is in the keyring.
 [PASS] There is a valid email id defined for the server key.
 [PASS] The public key can be used to encrypt a message.
 [PASS] The private key can be used to sign a message.
 [PASS] The public and private keys can be used to encrypt and sign a message.
 [PASS] The private key can be used to decrypt a message.
 [PASS] The private key can be used to decrypt and verify a message.
 [PASS] The public key can be used to verify a signature.
 [PASS] The server public key format is Gopengpg compatible.
 [PASS] The server private key format is Gopengpg compatible.

 Application configuration

 [PASS] Using latest passbolt version (4.0.2).
 [PASS] Passbolt is configured to force SSL use.
 [PASS] App.fullBaseUrl is set to HTTPS.
 [PASS] Selenium API endpoints are disabled.
 [PASS] Search engine robots are told not to index content.
 [INFO] The Self Registration plugin is enabled.
 [INFO] Registration is closed, only administrators can add users.
 [PASS] The deprecated self registration public setting was not found in /etc/passbolt/passbolt.php.
 [WARN] Host availability checking is disabled.
 [HELP] Make sure this instance is not publicly available on the internet.
 [HELP] Or set the PASSBOLT_EMAIL_VALIDATE_MX environment variable to true.
 [HELP] Or set passbolt.email.validate.mx to true in /etc/passbolt/passbolt.php.
 [PASS] Serving the compiled version of the javascript app.
 [WARN] Some email notifications are disabled by the administrator.

 JWT Authentication

 [PASS] The JWT Authentication plugin is enabled
 [PASS] The /etc/passbolt/jwt/ directory is not writable.
 [PASS] A valid JWT key pair was found

 SMTP Settings

 [PASS] The SMTP Settings plugin is enabled.
 [PASS] SMTP Settings coherent. You may send a test email to validate them.
 [PASS] The SMTP Settings source is: database.
 [WARN] The SMTP Settings plugin endpoints are enabled.
 [HELP] It is recommended to disable the plugin endpoints.
 [HELP] Set the PASSBOLT_SECURITY_SMTP_SETTINGS_ENDPOINTS_DISABLED environment variable to true.
 [HELP] Or set passbolt.security.smtpSettings.endpointsDisabled to true in /etc/passbolt/passbolt.php.

 [PASS] No error found. Nice one sparky!
14

To me it sounds like a database integrity issue. You could try this and see if it outputs anything:

sudo su -s /bin/bash -c "/usr/share/php/passbolt/bin/cake passbolt cleanup --dry-run" www-data

15 
sudo su -s /bin/bash -c "/usr/share/php/passbolt/bin/cake passbolt cleanup --dry-run" www-data

     ____                  __          ____
    / __ \____  _____ ____/ /_  ____  / / /_
   / /_/ / __ `/ ___/ ___/ __ \/ __ \/ / __/
  / ____/ /_/ (__  |__  ) /_/ / /_/ / / /
 /_/    \__,_/____/____/_.___/\____/_/\__/

 Open source password manager for teams
-------------------------------------------------------------------------------
 Cleanup shell (dry-run)
-------------------------------------------------------------------------------
No issue found, data looks squeaky clean!
16

I have been able to narrow it down to this ig:

sudo su -s /bin/bash -c "/usr/share/php/passbolt/bin/cake passbolt datacheck" www-data

...
[FAIL] Data integrity for Gpgkeys.
  [FAIL] Can encrypt: 27/33
    [PASS] Encryption success for key 3719B928E13254695B03A2B31F5028849DEFDAC2
    [PASS] Encryption success for key 23C5EE6D28D22C66F498CD0383E2CF08BFECAC95
    [PASS] Encryption success for key 24FF9DC6C482E649FD38FFD2EE751119D4ED30F2
    [PASS] Encryption success for key F7D6FF56C2F752DA2949BBA8AEB9F9D5B7664AF5
    [PASS] Encryption success for key F2223EB2957E382339E1A49AA4916F43B84480DE
    [PASS] Encryption success for key 8B3A56EF416B9E38A9BA704BE6101194869661B3
    [PASS] Encryption success for key 0C483CAD8BE48DBD190A529F4A761A465CE6DC2E
    [FAIL] Failed to encrypt with key 7BA18F137CAF2AD47602F5D70D1465FF8E54F8C4Q. Could not import the user OpenPGP key.
    [PASS] Encryption success for key 2BB6902D2366352DCBEC38C7BDA765C7583AAE6D
    [FAIL] Failed to encrypt with key E58599258FC30BD443918E7A75272900CA7E0DE3Q. Could not import the user OpenPGP key.
    [PASS] Encryption success for key 029D0B9553843AF2FEC6D04112104BB3F0ED177C
    [PASS] Encryption success for key F772FBC0999C497E378B5E0E3B9C67AD34D7E380
    [PASS] Encryption success for key AEC7B4C8BFABA695707B64E12286323BB727CD89
    [PASS] Encryption success for key 9E476C0BC8244BAB4E742FD1BFC7673D610E83E5
    [PASS] Encryption success for key 2D59573CC9E09711A3C381FD30F2FC6025BD1CF5
    [FAIL] Failed to encrypt with key 9ED4182C336A03DCBB3928C1F4A098D575F13C2CQ. Could not import the user OpenPGP key.
    [PASS] Encryption success for key 3A71E400AD0602A576B3161184810C866067CC1F
    [PASS] Encryption success for key 33572076314DA059FA7508AFD71FC5C1F120CF23
    [PASS] Encryption success for key 74EEAD0130C6FEE9E01589225C97DE21425F4CE0
    [FAIL] Failed to encrypt with key 1EB47508C31498F0B580438D671BF639FFEA1821Q. Could not import the user OpenPGP key.
    [PASS] Encryption success for key 5DE165BF83F4516012B7B38A3B5F2393CD881968
    [FAIL] Failed to encrypt with key 4B9ADB17A61B5824CB03E69233C654DE7BEF8349Q. Could not import the user OpenPGP key.
...
1 Like
17

After checking the database with the keys that have issues, I have narrowed it down to only 5 users.

What are the next steps to check what is wrong with these keys that does not allow the users to login?

18

This is not a valid fingerprint, so I’m not sure what is going on there.
I would try to get the keys from the database, load them manually in gnupg (or something else like mailvelope) and see what is wrong with them. Are they expired? Are they using unsupported algorithm? etc.

19

I will double check that as well,

All the users have their private keys with them, is there a way to update their key pair? even if it’s a db operation or admin only operation.

20 
All the users have their private keys with them, is there a way to update their key pair?

It depends on the udpate needed. If it’s an update of the key expiration date, yes it’s possible. If it’s a change in the key encryption algorithm i’d say it’s not possible (I’ve not tried).