(not an installation issue, this setup has been running for over 2 years - but as there is no category for general problem reports I posted in “installation issues”)
We can no longer add some users to any groups on our passbolt pro installation. The process always fails with “NetworkError when attempting to fetch resource.”
I haven’t found any similarities between the users that won’t work, i.e. I found 2 (out of 28) users by now that always fail for any group - one of them was created a few days ago, the other in November 2022.
I provide relevant information about my server (component names and versions, etc.)
passbolt 4.1.3
php 8.2.7 (also tried downgrading to version 8.1.20)
nginx 1.24.0 (on host and reverse proxy)
OS: FreeBSD 13.2-RELEASE-p1
openssl v1.1.1v
I provide a copy of my logs and healthcheck
Healthcheck shows all “PASS”; except the always broken “…config/jwt directory should not be writeable” (which it is not) and 2 warnings about email notifications and smtp plugin endpoints being disabled.
I describe the steps I have taken to trouble shoot the problem
As there are absolutely zero log-entries by passbolt about the event, I haven’t been abley to diagnose this properly.
Nginx (on host & reverse proxy), php-fpm and mysql logs are also clean.
I describe the steps on how to reproduce the issue
add an existing user to an existing group and hit ‘save’
the progress bar reaches 96%, then it stalls
after several seconds (timeout) “NetworkError when attempting to fetch resource.” is displayed
I recently updated from 4.1.0 to 4.1.3, which went flawless without any errors 'as usual’™
Reverting the latest pkg upgrades also didn’t help as well as switching back from php8.2 to php8.1
One of the users I can’t add to any group was created 6 days ago (but still on version 4.1.0) and just activated his account yesterday.
I first thought it might be the ‘ß’ in the username, but changing that to ‘ss’ didn’t solve the problem and we have users with umlauts that also don’t show any error. The second user I found also doesn’t have any special characters in its name; so that theory was quickly discarded.
I just tred to create a new test-user but that also fails with “NetworkError when attempting to fetch resource.”, so this seems to be a regression somewhere between 4.1.0 and 4.1.6?
Due to a wrong hard-coded path in utils.sh (‘/bin/bash’ - bash is never available in the base system on any BSD (or other unix like solaris/illumos for that matter)) - the “status-report” script doesn’t work under FreeBSD:
# bin/status-report www
su: unknown login: /bin/bash
So here’s the output of the 3 commands that this script tries to run:
root@passbolt:/usr/local/www/passbolt # su www -c 'bin/cake passbolt healthcheck'
____ __ ____
/ __ \____ _____ ____/ /_ ____ / / /_
/ /_/ / __ `/ ___/ ___/ __ \/ __ \/ / __/
/ ____/ /_/ (__ |__ ) /_/ / /_/ / / /
/_/ \__,_/____/____/_.___/\____/_/\__/
Open source password manager for teams
-------------------------------------------------------------------------------
The command should be executed with the same user as your web server. By instance:
su -s /bin/bash -c "/usr/local/www/passbolt/bin/cake COMMAND" HTTP_USER
where HTTP_USER match your web server user: www-data, nginx, apache, http
Healthcheck shell
-------------------------------------------------------------------------------
Environment
[PASS] PHP version 8.2.7.
[PASS] PCRE compiled with unicode support.
[PASS] The temporary directory and its content are writable and not executable.
[PASS] The logs directory and its content are writable.
[PASS] GD or Imagick extension is installed.
[PASS] Intl extension is installed.
[PASS] Mbstring extension is installed.
Config files
[PASS] The application config file is present
[PASS] The passbolt config file is present
Core config
[PASS] Debug mode is off.
[PASS] Cache is working.
[PASS] Unique value set for security.salt
[PASS] Full base url is set to https://passbolt.autogassner.de
[PASS] App.fullBaseUrl validation OK.
[PASS] /healthcheck/status is reachable.
SSL Certificate
[PASS] SSL peer certificate validates
[PASS] Hostname is matching in SSL certificate.
[PASS] Not using a self-signed certificate
Database
[PASS] The application is able to connect to the database
[PASS] 49 tables found
[PASS] Some default content is present
[PASS] The database schema up to date.
GPG Configuration
[PASS] PHP GPG Module is installed and loaded.
[PASS] The environment variable GNUPGHOME is set to /usr/local/www/.gnupg.
[PASS] The directory /usr/local/www/.gnupg containing the keyring is writable by the webserver user.
[PASS] The server OpenPGP key is not the default one
[PASS] The public key file is defined in /usr/local/www/passbolt/config/passbolt.php and readable.
[PASS] The private key file is defined in /usr/local/www/passbolt/config/passbolt.php and readable.
[PASS] The server key fingerprint matches the one defined in /usr/local/www/passbolt/config/passbolt.php.
[PASS] The server public key defined in the /usr/local/www/passbolt/config/passbolt.php (or environment variables) is in the keyring.
[PASS] There is a valid email id defined for the server key.
[PASS] The public key can be used to encrypt a message.
[PASS] The private key can be used to sign a message.
[PASS] The public and private keys can be used to encrypt and sign a message.
[PASS] The private key can be used to decrypt a message.
[PASS] The private key can be used to decrypt and verify a message.
[PASS] The public key can be used to verify a signature.
[PASS] The server public key format is Gopengpg compatible.
[PASS] The server private key format is Gopengpg compatible.
Application configuration
[PASS] Using latest passbolt version (4.1.3).
[PASS] Passbolt is configured to force SSL use.
[PASS] App.fullBaseUrl is set to HTTPS.
[PASS] Selenium API endpoints are disabled.
[PASS] Search engine robots are told not to index content.
[INFO] The Self Registration plugin is enabled.
[INFO] Registration is closed, only administrators can add users.
[PASS] The deprecated self registration public setting was not found in /usr/local/www/passbolt/config/passbolt.php.
[WARN] Host availability checking is disabled.
[HELP] Make sure this instance is not publicly available on the internet.
[HELP] Or set the PASSBOLT_EMAIL_VALIDATE_MX environment variable to true.
[HELP] Or set passbolt.email.validate.mx to true in /usr/local/www/passbolt/config/passbolt.php.
[PASS] Serving the compiled version of the javascript app.
[WARN] Some email notifications are disabled by the administrator.
JWT Authentication
[WARN] The JWT Authentication plugin is disabled
[HELP] Set the environment variable PASSBOLT_PLUGINS_JWT_AUTHENTICATION_ENABLED to true
SMTP Settings
[PASS] The SMTP Settings plugin is enabled.
[PASS] SMTP Settings coherent. You may send a test email to validate them.
[PASS] The SMTP Settings source is: database.
[WARN] The SMTP Settings plugin endpoints are enabled.
[HELP] It is recommended to disable the plugin endpoints.
[HELP] Set the PASSBOLT_SECURITY_SMTP_SETTINGS_ENDPOINTS_DISABLED environment variable to true.
[HELP] Or set passbolt.security.smtpSettings.endpointsDisabled to true in /usr/local/www/passbolt/config/passbolt.php.
[PASS] No error found. Nice one sparky!
root@passbolt:/usr/local/www/passbolt # su www -c 'bin/cake passbolt cleanup --dry-run'
____ __ ____
/ __ \____ _____ ____/ /_ ____ / / /_
/ /_/ / __ `/ ___/ ___/ __ \/ __ \/ / __/
/ ____/ /_/ (__ |__ ) /_/ / /_/ / / /
/_/ \__,_/____/____/_.___/\____/_/\__/
Open source password manager for teams
-------------------------------------------------------------------------------
Cleanup shell (dry-run)
-------------------------------------------------------------------------------
11 issues found in table Passbolt/Folders.FoldersRelations (duplicated folders relations)
11 issues detected, please re-run without --dry-run to fix.
root@passbolt:/usr/local/www/passbolt # su www -c 'bin/cake passbolt datacheck --hide-success-details'
____ __ ____
/ __ \____ _____ ____/ /_ ____ / / /_
/ /_/ / __ `/ ___/ ___/ __ \/ __ \/ / __/
/ ____/ /_/ (__ |__ ) /_/ / /_/ / / /
/_/ \__,_/____/____/_.___/\____/_/\__/
Open source password manager for teams
-------------------------------------------------------------------------------
Data check shell
[PASS] Data integrity for AuthenticationTokens.
[PASS] Can validate: 6548/6548
[PASS] Data integrity for Comments.
[PASS] Can validate: 6/6
[PASS] Data integrity for Favorites.
[PASS] Can validate: 0/0
[PASS] Data integrity for Gpgkeys.
[PASS] Can encrypt: 28/28
[PASS] Pass validation service checks: 28/28
[PASS] Entity data and armored key data matches: 28/28
[PASS] Is not expired: 28/28
[PASS] Is armored key format valid: 28/28
[PASS] Data integrity for Groups.
[PASS] Can validate: 15/15
[PASS] Data integrity for Profiles.
[PASS] Can validate: 38/38
[PASS] Data integrity for Resources.
[PASS] Can validate: 644/644
[PASS] Data integrity for Secrets.
[PASS] Can validate: 2745/2745
[PASS] Data integrity for Users.
[PASS] Can validate: 38/38
I let the ‘cleanup’ script fix the issues with the Passbolt/FoldersRelations, but this didn’t fix our problem:
I have created an internal ticket for this issues (PB-25799), since you are using Passbolt PRO could you contact us at support@passbolt.com linking this post? It will be easier for the follow-ups.
Thank you. I’m currently digging a bit deeper by incrementally restoring/rolling back the passbolt installation and/or packages and/or the whole jail from snapshots (while keeping the current database), so I might be able to find the last working state.
I’ll collect some more data from those attempts and will contact you via mail.
The progress bar reaches 96% at a reasonable time, then stalls for ~30 seconds until the “unexpected error” message is shown.
I don’t have any errors logged at the local or reverse proxy nginx - both ‘access.log’ show successful delivery of the ‘app/groups’ or ‘app/users’ page, the error.log remains silent during the whole process.
Also, that whole setup (including the reverse proxy) was running for >2 years now, the whole nginx config hasn’t been touched for months and the ‘server’ block for passbolt very likely since it has been set up…
Could you share the passbolt logs? Usually they are located in /var/log/passbolt/error.log but I can’t tell on FreeBSD since we are not officially supporting it.
Also, the payload from the API requests could give more details here, depending from your browser you could:
Sorry for the long silence.
The problem seems related to our reverse proxy setup. The proxy jail was migrated to another host (i.e. moving the config to a newly set up jail) and it somehow triggers this error with Passbolt.
Using the old jail (even on the same host) solves the problem immediately.
Both jails are configured identical (network interfaces/IPs) and use the same nginx version and config. I haven’t found the time yet to further troubleshoot this, so I’m just using the ‘old’ jail in the meantime…
So in essence, I’m not so sure any more if this really is a problem with Passbolt, although anything else behind the reverse proxy works just fine as is to be expected if using the same OS and package versions and config…
