Some tables are already present in the database,

Checklist
[ ] I have read intro post: About the Installation Issues category
[ ] I have read the tutorials, help and searched for similar issues
[ ] I provide relevant information about my server (component names and versions, etc.)
[ ] I provide a copy of my logs and healthcheck
[ ] I describe the steps I have taken to trouble shoot the problem
[ ] I describe the steps on how to reproduce the issue

I am trying to move my docker to another server…
I am using Docker-compose
i get this problem the make passbolt doesn’t start:

passbolt_1 | Some tables are already present in the database, a new installation would override existing data.
passbolt_1 | Please use --force to proceed anyway.

passbolt_1 | PDOException: SQLSTATE[42S01]: Base table or view already exists: 1050 Table ‘folders’ already exists in /var/www/passbolt/vendor/robmorgan/phinx/src/Phinx/Db/Adapter/PdoAdapter.php:184
passbolt_1 | Stack trace:

How can I fix that:

I have moved the docker folder to the new server. I have moved the certification.
docker-compose up -d
69a10fdf8d04 passbolt/passbolt:latest-pro “docker-php-entrypoi…” 9 minutes ago Exited (1) 6 minutes ago passbolt_docker_passbolt_1
e0bf6cd29adc mariadb:10.3 “docker-entrypoint.s…” 9 minutes ago Up 6 minutes 127.0.0.1:3306->3306/tcp passbolt_docker_db_1

although passbolt doesn’t start i restored the database.
but the issue is
passbolt_1 | == 20191119092944 V2130AddFoldersTable: migrating
passbolt_1 | PDOException: SQLSTATE[42S01]: Base table or view already exists: 1050 Table ‘folders’ already exists in /var/www/passbolt/vendor/robmorgan/phinx/src/Phinx/Db/Adapter/PdoAdapter.php:184
passbolt_1 | Stack trace:

@Ahmad hello, seems like you are using passbolt pro, you should get in touch with support@passbolt.com for help.

“Table are already present” error generally appears when you have a mix match between passbolt versions (software, database content, database structure). For example this can happen when you reloaded content from a previous versions from backups in a database schema that was already migrated.

Long story short, you have the database structure of a new version, but you miss some of the content (such as which migration have been run). These issues are hard to fix as you need to manually match the structure and recreate database migrations table content.

Another way to fix this is to export the data that you have, check what is the highest migration version number in the phinx table (which list the migrations), and install this older version of passbolt, run the initial migration, load the data. Then you can upgrade again to the latest version.

That seems right
my old version is 2.12.1 and now without meaning I have 2.13.1

Can you help me with downgrading?
1- docker-compose down
then I change in docker-compose-pro.yml
from
image: passbolt/passbolt:latest-pro
to
image: passbolt/passbolt:2.12.1
2- then docker-compose up
Is that right?

@Ahmad in your scenario the best would actually to just edit the database manually and drop the following tables:

  • folders
  • folders_history
  • folders_relations
  • folders_relations_history

From there you should be able to finish the migrations again.

Hi.

I did that and that worked… THANKS
I did that in a new server. I am really afraid to do that in the production server. Is that safe to drop those tables. will I lose any information?

Can yo help me with the upgrading procedure from 2.12.1
and downgrading from 2.13.1 to 2.12.1

It’s not sufficient to downgrade the code, you need both the code from v2.12.1 and database structure from v2.12.1. We don’t have official instruction to downgrade, it depends on your case (e.g where is your database, etc.). Basically the process is take your v2.12 backup and install a v2.12 software on a clean DB, load your v2.12 backup data.

It’s hard to tell. Most likely you will only loose “Folders” information if at some point you were running v2.13.1 in production. Maybe you will have some other data issues if you were running previous version of the software against new database schema version. Or a mix of both depending on what you did. The best is load your production data into a staging environment, try it, see if it works.

To be honest it’s hard to advise without knowing which operations you did and in which order. I would need to know which version of the software you ran against which version of the database structure, and when was data loaded from v2.12 backup, then look at the log to check what users did during that time, etc.

Database structure, data and software version are meant to be in sync and are not always backward/forward compatible, as you experienced. You can always run the data cleanup tool afterwards to check for consistency issue:

./bin/cake passbolt cleanup --dry-run

Database structure, data and software version are meant to be in sync

Can you please check if I have the right docker-compose-pro.yml
version: ‘3.4’
services:
db:
image: mariadb:10.3
env_file:
- env/mysql.env
volumes:
- database_volume:/var/lib/mysql
ports:
- “127.0.0.1:3306:3306”

passbolt:
image: passbolt/passbolt:latest-pro
#image: passboltlocal
tty: true
depends_on:
- db
env_file:
- env/passbolt.env
volumes:
- gpg_volume:/var/www/passbolt/config/gpg
- images_volume:/var/www/passbolt/webroot/img/public
- ./license:/var/www/passbolt/config/license
- /home/certs:/etc/ssl/certs/
tmpfs:
- /run
command: ["/usr/bin/wait-for.sh", “db:3306”, “–”, “/docker-entrypoint.sh”]
ports:
- 127.0.0.1:80:80
- 127.0.0.1:443:443

volumes:
database_volume:
gpg_volume:
images_volume:
certs:

Your docker-compose file looks ok, but it is difficult to say for sure.

Can you run the following query on your database and let us know the output?

SELECT * FROM phinxlog;

In the production server: not updated to 2.13 yet

20190106170300 | V280AdditionalEmailMigration | 2020-04-21 14:36:18 | 2020-04-21 14:36:18 | 0 |
| 20190106170301 | V280AdditionalFileStorageMigration | 2020-04-21 14:36:18 | 2020-04-21 14:36:18 | 0 |
| 20190106170302 | V280FileDirectoryPathsMigrations | 2020-04-21 14:36:18 | 2020-04-21 14:36:18 | 0 |
| 20190112124290 | V270AddActionsTable | 2020-04-21 14:36:18 | 2020-04-21 14:36:18 | 0 |
| 20190112124300 | V270AddActionLogsTable | 2020-04-21 14:36:18 | 2020-04-21 14:36:18 | 0 |
| 20190121111100 | V270AddEntitiesHistoryTable | 2020-04-21 14:36:18 | 2020-04-21 14:36:18 | 0 |
| 20190121121100 | V270AddPermissionsHistoryTable | 2020-04-21 14:36:18 | 2020-04-21 14:36:19 | 0 |
| 20190211124300 | V270AddSecretsHistoryTable | 2020-04-21 14:36:19 | 2020-04-21 14:36:19 | 0 |
| 20190221124300 | V270AddSecretAccessesTable | 2020-04-21 14:36:19 | 2020-04-21 14:36:19 | 0 |
| 20190327142000 | V280AddTagsIndexesMigrations | 2020-04-21 14:36:19 | 2020-04-21 14:36:19 | 0 |
| 20190512115400 | V2100AddOrganizationSettingsTable | 2020-04-21 14:36:19 | 2020-04-21 14:36:19 | 0 |
| 20190623143400 | V2110ExtendKeyIdSizeField | 2020-04-21 14:36:19 | 2020-04-21 14:36:19 | 0 |
| 20190923103000 | V2120UpdateEmailQueue | 2020-04-21 14:36:19 | 2020-04-21 14:36:29 | 0 |
| 20191119160000 | V2120DropUnusedTables | 2020-04-21 14:36:29 | 2020-04-21 14:36:29 | 0 |
±---------------±---------------------------------------±--------------------±--------------------±-----------+

in the test server:
updated to 2.13.1 , I don’t know how the updating just done.

20190106170301 | V280AdditionalFileStorageMigration | 2020-04-21 14:36:18 | 2020-04-21 14:36:18 | 0 |
| 20190106170302 | V280FileDirectoryPathsMigrations | 2020-04-21 14:36:18 | 2020-04-21 14:36:18 | 0 |
| 20190112124290 | V270AddActionsTable | 2020-04-21 14:36:18 | 2020-04-21 14:36:18 | 0 |
| 20190112124300 | V270AddActionLogsTable | 2020-04-21 14:36:18 | 2020-04-21 14:36:18 | 0 |
| 20190121111100 | V270AddEntitiesHistoryTable | 2020-04-21 14:36:18 | 2020-04-21 14:36:18 | 0 |
| 20190121121100 | V270AddPermissionsHistoryTable | 2020-04-21 14:36:18 | 2020-04-21 14:36:19 | 0 |
| 20190211124300 | V270AddSecretsHistoryTable | 2020-04-21 14:36:19 | 2020-04-21 14:36:19 | 0 |
| 20190221124300 | V270AddSecretAccessesTable | 2020-04-21 14:36:19 | 2020-04-21 14:36:19 | 0 |
| 20190327142000 | V280AddTagsIndexesMigrations | 2020-04-21 14:36:19 | 2020-04-21 14:36:19 | 0 |
| 20190512115400 | V2100AddOrganizationSettingsTable | 2020-04-21 14:36:19 | 2020-04-21 14:36:19 | 0 |
| 20190623143400 | V2110ExtendKeyIdSizeField | 2020-04-21 14:36:19 | 2020-04-21 14:36:19 | 0 |
| 20190923103000 | V2120UpdateEmailQueue | 2020-04-21 14:36:19 | 2020-04-21 14:36:29 | 0 |
| 20191119092944 | V2130AddFoldersTable | 2020-07-17 12:20:06 | 2020-07-17 12:20:06 | 0 |
| 20191119092945 | V2130AddFoldersHistoryTable | 2020-07-17 12:20:06 | 2020-07-17 12:20:06 | 0 |
| 20191119160000 | V2120DropUnusedTables | 2020-04-21 14:36:29 | 2020-04-21 14:36:29 | 0 |
| 20191216092944 | V2130AddFoldersRelationsTable | 2020-07-17 12:20:06 | 2020-07-17 12:20:06 | 0 |
| 20191216092945 | V2130AddFoldersRelationsHistoryTable | 2020-07-17 12:20:06 | 2020-07-17 12:20:06 | 0 |
| 20200108135000 | V2130DropLegacyAnonymousUser | 2020-07-17 12:20:06 | 2020-07-17 12:20:06 | 0 |
| 20200205135000 | V2130AddResourcesFoldersRelations | 2020-07-17 12:20:06 | 2020-07-17 12:20:07 | 0 |
| 20200319135000 | V2130SoftDeleteGpgKeysForSoftDeletedUsers | 2020-07-17 12:20:07 | 2020-07-17 12:20:07 | 0 |
| 20200609192000 | V2130AddMissingFoldersIndexes | 2020-07-17 12:20:07 | 2020-07-17 12:20:07 | 0 |
±---------------±------------------------------------------±--------------------±--------------------±-----------+

I still need the updating procedures from 2,12,1 to 2.12.3

I guess you mean 2.13. Ok, so the error you posted at the top, is it coming from your production server? Or was it from your test server?

Can you on your production server check:

show tables;

If you have any “folders” table can you do:

select count(id) from folders;

±-----------------------------------+
| Tables_in_passbolt |
±-----------------------------------+
| account_settings |
| action_logs |
| actions |
| authentication_tokens |
| comments |
| directory_entries |
| directory_ignore |
| directory_relations |
| directory_reports |
| directory_reports_items |
| email_queue |
| entities_history |
| favorites |
| file_storage |
| gpgkeys |
| groups |
| groups_users |
| organization_settings |
| passbolt_account_settings_phinxlog |
| passbolt_directory_sync_phinxlog |
| passbolt_tags_phinxlog |
| permissions |
| permissions_history |
| phinxlog |
| profiles |
| resources |
| resources_tags |
| roles |
| secret_accesses |
| secrets |
| secrets_history |
| tags |
| user_agents |
| users |
±--------

ERROR 1146 (42S02): Table ‘passbolt.folders’ doesn’t exist

@Ahmad from the look of it you should be able to upgrade the production database without the issue you faced on your test server, since the schema in your test environment and production were not the same.

If you want to be sure, pick up the database data and structure from the production database, load it in your test server and run the migration.

If I take backup now from production and restore it to the test, then docker-compose restart. I get the same problem that about table folders…

I need the steps to upgrade the production docker…

This is because you are loading the data in an already migrated schema. You need to make a backup of both the schema (create tables statements) and the data (insert statements). You need to load this in a clean test database without any tables (drop the tables first). From there you will be able to have the same database (schema and data) that your production environment, and test that the migration works as intended.

I don’t know exactly how to do that… I just take backup via mysqldump and the restore it… :slight_smile:
If you can help with more details about: make a backup of both the schema (create tables statements) and the data (insert statements). You need to load this in a clean test database without any tables (drop the tables first).
I would be very thankful…

@Ahmad ok I understand your problem. Basically when you do a mysqldump it will contain the schema and data and do a “drop table if exist”, but it will not drop tables that are “not there” in the mysqldump but already in your test database. So it will only drop tables that are in production, not more.

So the easiest is just to cleanup your test database completely prior to loading your data from your production mysqldump, so something like:

drop database passbolt_test;
create database passbolt_test COLLATE utf8mb4_unicode_ci;

And then only you load your data from the dump.