Thanks for the suggestion, I watched your presentation and some of the concepts highlighted in your plugin are matching the direction we are currently taking to write tests, albeit with a different implementation.
To give you a little bit of context, for historical reason, the test workflow rely on a plugin called “passbolt_data”, that generates a “complete” data set to be used to prove the permission system. See. https://github.com/passbolt/passbolt_test_data . This data is also used to create a data set to to place the application in a given state used fro demo or integration tests (as in selenium tests). Fixtures were/can be re-generated automatically from this data set.
We have also developed additional data sets for other “simpler” scenarios. We are now moving in a direction where the data used for such scenarios are defined by the test itself using helper functions in the test class itself or via traits. This is mostly for loading test data, as this is our primary pain point, and not schemas. It should be possible to use dedicated classes instead, to tidy things up. I must say your solution is quite elegant in this regards.
I have some reservations the way schema are loaded thought. If I understood correctly the schema is loaded from the entity/table definitions? I’m not sure we’d want that rather than the more “traditional” way, as it would prevent us from testing certain scenarios touching on continuous / backward compatible database migrations (e.g. scenario where you would have the DB schema in one state and the entities in another, for example say you are reducing/increasing the size of a field or renaming a field, see. https://en.wikipedia.org/wiki/Evolutionary_database_design).
CakePHP fixtures handle the test DB schema in a parallel manner to the default DB. On the one hand you will write migrations for your defautl DB. And on the other hand you either hard coded describe the schema structure in your fixtures, or meme the default DB. The later is simpler, but it forces you to have two DBs. And in CI tools, you will have to run the migrations on your default DB, and the fixtures meme the default DB. So why not running migrations directly on the test DB?
With the CakePHP Fixture Factories, the schema of both default and test DB are handled exactly the same way. You do not need a default DB anymore to run your tests. Tables are not droped between test suites, which speeds up your tests. And migrations are part of the whole testing process.
The package includes a tool that I call the Migrator.
The migrations are run in tests/bootstrap.php. As long as your schema does not change, no migrations are re-run. This speeds up the tests, but most importantly, the process being identical to the default process, your migrations get tested. If you had a ticket where a migration added a corrupted field, say is_published_oups_typo, and you wrote a test related to that ticket using that new field is_published, the test is going to fail because the field is_published does not exist. Your migration got tested before it went live.
You can also write migration specific tests, e.g. if you increased the size of a field in a table, you can simply ensure in that an entity with that field of a given length is properly saved.
So I suppose, the package is much evolutionary DB design friendly.
Note also that the package is compatible with the traditional fixtures. Both approaches can live together, you do not have to get rid of your fixtures and of your home made solution.
Long live TDD, and let me know if you have any other concerns!