Not all logs are being removed by action_logs_purge

Installation Issues
1

Checklist
I have read intro post: https://community.passbolt.com/t/about-the-installation-issues-category/12
I have read the tutorials, help and searched for similar issues
I provide relevant information about my server (component names and versions, etc.)

$ apt list --installed passbolt-ce-server
Listing... Done
passbolt-ce-server/buster,now 4.11.1-1 all [installed]

I provide a copy of my logs and healthcheck
I describe the steps I have taken to trouble shoot the problem
I describe the steps on how to reproduce the issue

Problem description

We have collected quite a lot of rows in the action_logs table of our passbolt database.
We’ve used cake with the following command to purge the logs:

# /usr/share/php/passbolt/bin/cake passbolt action_logs_purge -r 30d

We’ve noticed that we still have many logs from before the given retention policy.

Looking at the mysql query logs while running the above command we noticed that only the action logs with the following action ids are purged:

select name from actions WHERE id in ('4d2d5c43-a76a-5d4a-8f47-2b1f0268e3d6','11ed2b2c-9ade-50b3-9bbb-68232d5dc5d7','f52ecf6c-8e82-5f3d-ab73-f6df46eb71b5','94c6e76f-9fd8-51bd-8e0c-2bfd1864d139','40cbf76f-74e9-597d-9136-0cb2685cd215','e42b232e-6ddb-56c6-933e-f20c95cdce60','9b473b19-0e3e-5efb-be19-8a1719843762','748dcd10-7d15-5498-9aa6-d26de348ff02','69692f38-cbbc-53c7-a44d-68164b57a091','9f7e766d-a30e-50d5-b16d-5cdc474fd1e8','7f6e3578-fcc7-590e-afc0-7937308cac94','1cd53591-cb6b-5b03-b0be-05a54644263d','ab13b3e0-eae8-5099-909f-8582a3c2ddae','caea12fd-e0f8-5d68-aa19-d7b25119740e','a8b75aae-cb86-5fda-a8b3-d36ae6155023','f0c5d1cd-b3e3-5494-8fb7-73a427790ce2','3cffe6ef-ea4c-5bc3-869b-945f26e2601a','08903e2d-1cb6-5908-8dbc-25b7a54e6bc4','1035f824-fe76-5a69-b02c-9aa60a31858c','dad1db3f-4fbc-5b3a-9db2-b7413885c30c','9a24eac2-58f4-576c-869b-82aaf44c435e','6d616537-c449-589d-bebe-b2d5883e9d35','30259dec-c224-509d-b742-96e2a6a85475','9caaba03-49d2-5273-8097-e278234e71e0','31f74e04-d9bb-592b-aa45-ba6682d3316d','1df624ae-eea2-5055-a202-d1068feefe04','eb53befb-c89c-5bea-99a8-0376f8dedc59','e2aa01a9-84ec-55f8-aaed-24ee23259339','d77de180-4626-54dd-80fb-a095ecc22aac','c506210f-7866-5691-8fc1-58772e8f49f1','83bb8bd8-2006-5546-a3bb-9319ae6e8f9d','30237a18-fd12-5935-b1be-1e2a62ccb71d','bef9f3ca-86ef-5c6a-9b38-320e03ceb5df','7df37cb5-cfb9-57c2-a7a5-b65c9f573de0','10807e9e-d525-5acc-b05d-cccbbd252a93','941bab2b-038f-5b6d-be9b-6fb10ec9d675','3efcc1ec-5cc1-5d2a-a16e-234f23b70719','69894f9f-5349-5289-ba93-8fb002ac0963','afd95c6c-2122-5818-ab09-d2e3a087e79d','058f384b-6685-55ac-8e82-9b0579a4e210','dcb2e27e-dd2e-5215-9e4e-e0108519a441','4be6e190-6e62-5b23-a392-029ce9871d9c','d7bc9044-a64e-5421-a4d7-7a94eaa39d37','881ab948-e40f-5a72-91aa-54b442270029','af9aa2c6-7355-514d-a4a0-3e74de4c0fdb','4dd4a164-b4e1-57eb-ac2e-4a223810323c');
+------------------------------------+
| name                               |
+------------------------------------+
| AccountSettingsIndex.index         |
| AuthVerify.verifyGet               |
| CommentsView.view                  |
| FoldersIndex.index                 |
| FoldersView.view                   |
| GpgkeysIndex.index                 |
| GroupsIndex.index                  |
| HealthcheckIndex.index             |
| Home.apiApp                        |
| Home.apiExtApp                     |
| MfaOrgSettingsGet.get              |
| MfaSetupSelectProvider.get         |
| MfaVerifyAjaxError.get             |
| NotificationOrgSettingsGet.get     |
| PermissionsView.viewAcoPermissions |
| ResourcesIndex.index               |
| ResourcesView.view                 |
| ResourceTypesIndex.index           |
| ResourceTypesView.view             |
| RolesIndex.index                   |
| SettingsIndex.index                |
| Share.dryRun                       |
| ShareSearch.searchArosToShareWith  |
| shell                              |
| ThemesIndex.index                  |
| TotpSetupGet.get                   |
| TotpVerifyGet.get                  |
| UsersIndex.index                   |
| UsersView.view                     |
+------------------------------------+
29 rows in set (0.001 sec)

The logs for the remaining 49 action types are still present in the action_logs table.

We ended up manually deleting rows from the action_logs table. (delete from action_logs where created <= '202X-XY-XY 00:00:00')

Why doesn’t the above cake command doesn’t remove all action_logs according to the given retention policy?

Looking at passbolt_api/plugins/PassboltCe/Log/src/Service/ActionLogs/ActionLogsPurgeService.php this behaviour seems to be configurable (passbolt.plugins.log.config.blackList).

Could someone enlighten me about the functinoality of action_logs_purge?

2

Hello @hunter2,

The logging system of Passbolt is meant to keep accurate tracks of all the actions performed by the users on the system. The action_logs_purge command has been developed to purge the action_logs table in order to free some space in DB as it is bound to become oversized through time.

This command is meant to purge the table from the entries that are not relevant to the admins and the users by only keeping the ones with particular relevance to security (like resource creation, permission changes …).

Cheers,
Gérold.

3

The purge command is selective by design. It doesn’t delete all logs older than the retention period, but rather targets specific types of logs while preserving others. Here’s what it does:

  • It only purges logs for specific action types listed in the getActionList() method (the 29 action types you saw in your query)
  • It preserves any action logs that have references in the entities_history table, regardless of age. This is intentional to maintain a complete audit trail of data changes.
4

Hello @max

Thanks for the clarification.

Regards

1 Like