After Ubuntu 18.04 ec2 restart the public Ip changed and throws internal server error

dheerajk · May 24, 2023, 4:49pm

Checklist
I have read intro post: About the Installation Issues category
I have read the tutorials, help and searched for similar issues
I provide relevant information about my server (component names and versions, etc.)
I provide a copy of my logs and healthcheck
I describe the steps I have taken to trouble shoot the problem
I describe the steps on how to reproduce the issue

Hello, We have a community edition installed on an Ubuntu ec2 and recently the page was not loading properly. So i restarted the EC2 which has a public IP that changed after restart. We use a friendly DNS name ( hosted zone in Route 53) after the public Ip changed I have updated the Route53 entry pointing to new IP. when I reach the URL i see Internal server error. The nginx process is up and so does mysql

Health Check
sudo -H -u www-data bash -c “/var/www/passbolt/bin/cake passbolt healthcheck” www-data

     ____                  __          ____  
    / __ \____  _____ ____/ /_  ____  / / /_ 
   / /_/ / __ `/ ___/ ___/ __ \/ __ \/ / __/ 
  / ____/ /_/ (__  |__  ) /_/ / /_/ / / /    
 /_/    \__,_/____/____/_.___/\____/_/\__/   

 Open source password manager for teams
---------------------------------------------------------------
 Healthcheck shell.....Exception: SQLSTATE[HY000] [1045] Access denied for user 'vault'@'localhost' (using password: YES) in [/var/www/passbolt/vendor/cakephp/cakephp/src/Database/Driver.php, line 92]

this server was setup by my teammate who is no longer with the company. I am able to see the password for the user “vault” and the database name is “pb” . host is 127.0.0.1

In /var/www/passbolt/config/passbolt.php the ip is not hardcoded, its the friendly dns that is in the config

when I try to get in to the mysql as root i get error
sudo mysql

ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: NO)

also tried login as below still error
mysql -u passboltadmin -p passboltdb -h 127.0.0.1

Enter password:

ERROR 1045 (28000): Access denied for user 'passboltadmin'@'localhost' (using password: NO)

In the error.log it says the auth failed for user vault to the database

when i try the password thats in passbolt.php i get the same error
mysql -u vault -p -h localhost pb

Enter password:

ERROR 1045 (28000): Access denied for user 'vault'@'localhost' (using password: YES)

systemctl status mysqld

● mariadb.service - MariaDB 10.1.48 database server
   Loaded: loaded (/lib/systemd/system/mariadb.service; enabled; vendor preset: enabled)
   Active: active (running) since Tue 2023-05-23 19:55:13 UTC; 20h ago
     Docs: man:mysqld(8)
           https://mariadb.com/kb/en/library/systemd/
 Main PID: 1009 (mysqld)
   Status: "Taking your SQL requests now..."
    Tasks: 28 (limit: 2342)
   CGroup: /system.slice/mariadb.service
           └─1009 /usr/sbin/mysqld

May 23 19:55:12 ip-172-31-38-210 mysqld[1009]: 2023-05-23 19:55:12 139977202973824 [Note] /usr/sbin/mysqld (mysqld 10.1.48-MariaDB-0ubuntu0.18.04.1) starting as process 1009 ...
May 23 19:55:13 ip-172-31-38-210 /etc/mysql/debian-start[1048]: Upgrading MySQL tables if necessary.
May 23 19:55:13 ip-172-31-38-210 systemd[1]: Started MariaDB 10.1.48 database server.
May 23 19:55:13 ip-172-31-38-210 /etc/mysql/debian-start[1052]: /usr/bin/mysql_upgrade: the '--basedir' option is always ignored
May 23 19:55:13 ip-172-31-38-210 /etc/mysql/debian-start[1052]: Looking for 'mysql' as: /usr/bin/mysql
May 23 19:55:13 ip-172-31-38-210 /etc/mysql/debian-start[1052]: Looking for 'mysqlcheck' as: /usr/bin/mysqlcheck

garrett · May 24, 2023, 5:12pm

Hi @dheerajk Welcome to the forum!

Thank you for the helpful information. A few questions, if you know the answers:

is Docker involved at all?
are there other databases in mysql as well as the pb database?
can you speak the installation structure regarding folder locations? For example, is NGINX serving it from /var/www/html or another location?
Are there files in /etc/passbolt or does this directory exist?
which PHP is running? php -v

And can you access with: mysql -u passboltadmin -p?

Termindiego25 · May 25, 2023, 8:00am

If you have root password for mysql you can try mysql -u root -p instead of sudo mysql

dheerajk · May 25, 2023, 12:42pm

is Docker involved at all?
docker is not involved passbolt is running on a self hosted ubuntu ec2 server
are there other databases in mysql as well as the pb database?
No, only pb database
can you speak the installation structure regarding folder locations? For example, is NGINX serving it from /var/www/html or another location?

this is where nginx.conf is /etc/nginx/nginx.conf

root@ip-XX.XX.XX:/etc/nginx# ls -l

total 64

drwxr-xr-x 2 root root 4096 Dec 7 2020 conf.d

-rw-r–r-- 1 root root 1077 Apr 6 2018 fastcgi.conf

-rw-r–r-- 1 root root 1007 Apr 6 2018 fastcgi_params

-rw-r–r-- 1 root root 2837 Apr 6 2018 koi-utf

-rw-r–r-- 1 root root 2223 Apr 6 2018 koi-win

-rw-r–r-- 1 root root 3957 Apr 6 2018 mime.types

drwxr-xr-x 2 root root 4096 Jan 10 2020 modules-available

drwxr-xr-x 2 root root 4096 Dec 7 2020 modules-enabled

-rw-r–r-- 1 root root 1517 Dec 7 2020 nginx.conf

-rw-r–r-- 1 root root 180 Apr 6 2018 proxy_params

-rw-r–r-- 1 root root 636 Apr 6 2018 scgi_params

drwxr-xr-x 2 root root 4096 Nov 17 2022 sites-available

drwxr-xr-x 2 root root 4096 Dec 7 2020 sites-enabled

drwxr-xr-x 2 root root 4096 Nov 17 2022 snippets

-rw-r–r-- 1 root root 664 Apr 6 2018 uwsgi_params

-rw-r–r-- 1 root root 3071 Apr 6 2018 win-utf

If you have root password for mysql you can try mysql -u root -p instead of sudo mysql

I do have root password. I have the pem key file and login as ubuntu then i do sudo.

Thanks !

garrett · May 25, 2023, 3:41pm

Can you answer these as well, thanks.

Termindiego25 · May 26, 2023, 7:11am

Hello, I mean root password of database not root password of ssh. Maybe it has default credentials

dheerajk · May 26, 2023, 1:20pm

can you access with: mysql -u passboltadmin -p ?
i can but I donot have the password for this user
- Are there files in /etc/passbolt or does this directory exist?
  nope, there is no dir /etc/passbolt
  but its located here /var/www/passbolt
  /var/www/passbolt# ls -l
  total 524
  -rw-r–r-- 1 www-data www-data 136 Dec 7 2020 Berksfile
  -rw-r–r-- 1 www-data www-data 109 Dec 7 2020 Berksfile.lock
  -rw-r–r-- 1 www-data www-data 42850 Dec 7 2020 CHANGELOG.md
  -rw-r–r-- 1 www-data www-data 1302 Dec 7 2020 CONTRIBUTING.md
  -rw-r–r-- 1 www-data www-data 198 Dec 7 2020 Gemfile
  -rw-r–r-- 1 www-data www-data 17732 Dec 7 2020 Gemfile.lock
  -rw-r–r-- 1 www-data www-data 5308 Dec 7 2020 Gruntfile.js
  -rw-r–r-- 1 www-data www-data 34520 Dec 7 2020 LICENSE.txt
  -rw-r–r-- 1 www-data www-data 4583 Dec 7 2020 README.md
  drwxr-xr-x 2 www-data www-data 4096 Dec 7 2020 bin
  -rw-r–r-- 1 www-data www-data 9 Dec 7 2020 chefignore
  -rw-r–r-- 1 www-data www-data 3973 Dec 7 2020 composer.json
  -rw-r–r-- 1 www-data www-data 194882 Dec 7 2020 composer.lock
  dr-xr-x— 6 www-data www-data 4096 Dec 7 2020 config
  drwxr-xr-x 5 www-data www-data 4096 Dec 7 2020 debian
  -rw-r–r-- 1 www-data www-data 648 Dec 7 2020 index.php
  -rw-r–r-- 1 www-data www-data 1324 Dec 7 2020 kitchen.yml
  drwxrwxrwx 2 www-data www-data 4096 Feb 5 06:49 logs
  -rw-r–r-- 1 www-data www-data 137478 Dec 7 2020 package-lock.json
  -rw-r–r-- 1 www-data www-data 637 Dec 7 2020 package.json
  -rw-r–r-- 1 www-data www-data 1869 Dec 7 2020 phpunit.xml.dist
  drwxr-xr-x 3 www-data www-data 4096 Dec 7 2020 plugins
  drwxr-xr-x 16 www-data www-data 4096 Dec 7 2020 src
  drwxr-xr-x 7 www-data www-data 4096 Dec 7 2020 tests
  drwxr-xrwx 6 www-data www-data 4096 Dec 7 2020 tmp
  drwxrwxr-x 23 www-data www-data 4096 Dec 7 2020 vendor
  drwxr-xr-x 6 www-data www-data 4096 Sep 30 2021 webroot
which PHP is running? php -v
php -v
PHP 7.2.24-0ubuntu0.18.04.17 (cli) (built: Feb 23 2023 13:29:25) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.2.0, Copyright (c) 1998-2018 Zend Technologies
with Zend OPcache v7.2.24-0ubuntu0.18.04.17, Copyright (c) 1999-2018, by Zend Technologies

dheerajk · May 26, 2023, 1:22pm

nope, i donot have the root password or i dont even know the root username for database. The only creds I have for database is for user : vault that is saved in passbolt.php

Thanks !

garrett · May 26, 2023, 1:35pm

Thanks, these questions were to try to determine how old your version is. It’s old.

Being locked out of mysql can be resolved if you have root server access.

I was looking for a tutorial and How To Reset Your MySQL or MariaDB Root Password | DigitalOcean is along the lines of what you need.

Before step 4 I would do:
USE mysql; (switch to mysql database)
SELECT * FROM user; (to view users already in the database)

Make note of all of the users, like root, passbolt, passboltadmin, etc. Determine which user is doing what with, for example:
SHOW GRANTS for 'passbolt'@'localhost'

Since the healthcheck did not run, double check the passbolt.php config file in the project web root as it’s not clear which user is expected to access the db.

I think it was originally installed via source - and the instructions would have been like this and might help you understand the roles of the users: Passbolt Help | Install passbolt API from source

Once you get the user access squared away, try to run the healthcheck again.

dheerajk · May 31, 2023, 2:52pm

Hello, I tried resetting the password but when I switch to mysql at step 4 it didn’t work , password gets updated though but when i login it fails.

steps worked for me:

reset the root password following same exact steps :

then i tried resetting the password for user “vault”, password gets updated but it doesnt work. so I created a new user ( admin permissions) updated passbolt.php under config directory. now passbolt UI is back. but health check is giving some cert error.

and I restarted the server because after login it shows blank like i attached in the image, just a reminder the public Ip changed and in the app base url i have the friendly url so not sure where its complaining the health check needs to be updated. i checked the file passbolt.php there is nothing like health check

 ____                  __          ____  
/ __ \____  _____ ____/ /_  ____  / / /_

/ // / __ `/ / / __ / __ / / _/
/ / // ( | ) // / // / / /
// _,///./_//__/

Open source password manager for teams

Healthcheck shell

Environment

[PASS] PHP version 7.2.24-0ubuntu0.18.04.17.
[PASS] PCRE compiled with unicode support.
[PASS] The temporary directory and its content are writable.
[PASS] The public image directory and its content are writable.
[PASS] The logs directory and its content are writable.
[PASS] GD or Imagick extension is installed.
[PASS] Intl extension is installed.
[PASS] Mbstring extension is installed.

Config files

[PASS] The application config file is present
[PASS] The passbolt config file is present

Core config

[PASS] Debug mode is off.
[PASS] Cache is working.
[PASS] Unique value set for security.salt
[PASS] Full base url is set to https://pb.withmehealth.com
[PASS] App.fullBaseUrl validation OK.
[FAIL] Could not reach the /healthcheck/status with the url specified in App.fullBaseUrl
[HELP] Check that the domain name is correct in config/passbolt.php
[HELP] Check the network settings

SSL Certificate

[FAIL] SSL peer certificate does not validate
[FAIL] Hostname does not match when validating certificates.
[WARN] Using a self-signed certificate
[HELP] fopen(): php_network_getaddresses: getaddrinfo failed: Name or service not known
fopen(https://pb.withmehealth.com/healthcheck/status.json): failed to open stream: php_network_getaddresses: getaddrinfo failed: Name or service not known

Database

[PASS] The application is able to connect to the database
[PASS] 23 tables found
[PASS] Some default content is present
[PASS] The database schema up to date.

GPG Configuration

[PASS] PHP GPG Module is installed and loaded.
[PASS] The environment variable GNUPGHOME is set to /home/www-data/.gnupg.
[PASS] The directory /home/www-data/.gnupg containing the keyring is writable by the webserver user.
[PASS] The server gpg key is not the default one
[PASS] The public key file is defined in config/passbolt.php and readable.
[PASS] The private key file is defined in config/passbolt.php and readable.
[PASS] The server key fingerprint matches the one defined in config/passbolt.php.
[PASS] The server public key defined in the config/passbolt.php (or environment variables) is in the keyring.
[PASS] There is a valid email id defined for the server key.
[PASS] The public key can be used to encrypt a message.
[PASS] The private key can be used to sign a message.
[PASS] The public and private keys can be used to encrypt and sign a message.
[PASS] The private key can be used to decrypt a message.
[PASS] The private key can be used to decrypt and verify a message.
[PASS] The public key can be used to verify a signature.

Application configuration

[FAIL] This installation is not up to date. Currently using 2.13.5 and it should be v4.0.0.
[HELP] See. Passbolt Help | Update
[PASS] Passbolt is configured to force SSL use.
[PASS] App.fullBaseUrl is set to HTTPS.
[PASS] Selenium API endpoints are disabled.
[PASS] Search engine robots are told not to index content.
[PASS] Registration is closed, only administrators can add users.
[PASS] Serving the compiled version of the javascript app
[PASS] All email notifications will be sent.

4 error(s) found. Hang in there!

dheerajk · May 31, 2023, 2:54pm

I also see the passbolt version is outdated, if upgrading will fix the issue. how can i do it with out losing the data ?
Thanks !

garrett · May 31, 2023, 3:37pm

That’s great news! See Blank page with passbolt badge after authentication

… for recent example of same issue. Blank page is because an upgrade is needed.

dheerajk · May 31, 2023, 6:38pm

Hello Gerald, I have upgraded ubuntu to 20.04 and then followed the steps to upgrade passbolt

when i started i see nginx webpage which is good so I know its a virtual host issue i can fix it in nginx.conf

i have updated ‘fullBaseUrl’ value in /etc/passbolt/passbolt.php starts with https://

and also removed symbolic link for /etc/nginx/sites-available/default and also updated server name

still some issue in routing

current behavior :
http://url lands to nginx webpage and redirects to https://url

if i hit https://url it fails

if i hit http://URL/app/passwords redirects to https and says site cant be found

/etc/nginx/sites-available/nginx-passbolt.conf:

#
#  Passbolt.conf - Nginx configuration file to run the Passbolt software.
#

server {

  listen 80;
  listen [::]:80;

  # Managed by Passbolt
  server_name pb.withmehealth.com;

  client_body_buffer_size     100K;
  client_header_buffer_size   1K;
  client_max_body_size        5M;

  client_body_timeout   10;
  client_header_timeout 10;
  keepalive_timeout     5 5;
  send_timeout          10;

  root /usr/share/php/passbolt/webroot;
  index index.php;
  error_log /var/log/nginx/passbolt-error.log info;
  access_log /var/log/nginx/passbolt-access.log;

  # Managed by Passbolt
  # include __PASSBOLT_SSL__

  location / {
    try_files $uri $uri/ /index.php?$args;
  }

  location ~ \.php$ {
    try_files                $uri =404;
    include                  fastcgi_params;
    fastcgi_pass             unix:/run/php/__PHP_SOCK__;
    fastcgi_index            index.php;
    fastcgi_intercept_errors on;
    fastcgi_split_path_info  ^(.+\.php)(.+)$;
    fastcgi_param            SCRIPT_FILENAME $document_root$fastcgi_script_name;
    fastcgi_param            SERVER_NAME $http_host;
    fastcgi_param PHP_VALUE  "upload_max_filesize=5M \n post_max_size=5M";
  }

}

/etc/nginx/sites-available/nginx-passbolt.conf (END)

#################
i tried adding
listen 443;
listen [::]:443; but didnt help

in virtualhost i added include statement

include /etc/nginx/sites-enabled/*.conf;

Thanks !

garrett · May 31, 2023, 6:53pm

I changed your post so we can see the hashtags which are commenting out the __PASSBOLT_SSL__line. I believe if you follow the http to https steps on the help site it will create a separate SSL nginx file maybe at /etc/nginx/ssl? And then that file gets included.

dheerajk · June 1, 2023, 3:10pm

Hello, i see the file gets created /etc/ssl
drwxr-xr-x 2 root root 20480 May 31 19:34 certs

-rw-r–r-- 1 root root 10909 May 24 17:14 openssl.cnf

drwx------ 2 root root 4096 May 31 19:34 private

can i just leave this as http for now, i want to get in there and get few things, take backup and move to ubuntu 22.04
if you can give me instructions to use http only thats great.
Thanks !

garrett · June 1, 2023, 4:07pm

http only would be:

in passbolt.php config file change fullBaseUrl to begin with http://
also (same config file) check that passbolt.ssl.force is false or not set
if NGINX is redirecting 80 to 443 then remove, but I think you are indicating that is not currently functional

dheerajk · June 1, 2023, 9:18pm

It works finally with SSL. Thanks for the quick response.