TAF: open source framework comparing MariaDB, MySQL and PostgreSQL

Anyone choosing between MariaDB, MySQL and PostgreSQL for a new cluster, or trying to determine whether an upgrade has slowed queries, faces a common problem: published benchmarks are almost never reproducible because the configuration used is never published in full. The MariaDB Foundation aims to solve this with TAF, the Test Automation Framework developed by Jonathan Miller, who has been with the Foundation for about a year after nearly 27 years at Pervasive SQL and MySQL/Oracle.

What TAF does

TAF, the Test Automation Framework, has been downloaded over 400 times in the two weeks since version 4.0 was released, and it allows you to install MariaDB, MySQL and PostgreSQL side by side, run the same identical benchmark on each one, and pass the exact configuration to another user so they can reproduce the result. The key mechanism is what Miller calls "configuration provenance": whatever the source of the configuration actually used during the test, that is what ends up in the properties file generated at the end of the run. Anyone receiving that file installs TAF and the same database versions, runs it, and gets the same result.

The project is publicly available on GitHub (MariaDB/TAF repository), written in Perl to run without additional dependencies on any Linux distribution used for development, testing or production.

Recent releases

The development pace has been brisk in recent weeks: version 4.0, released on 28 September 2026, represents the biggest changes since the framework's inception and redesigns how workloads are organised, executed and reported on MariaDB, MySQL and now PostgreSQL. A few days later, on 5 October, all current test suites began supporting PostgreSQL, with the plugin integrating with TAF's workload engines allowing HammerDB, sysbench and any SQL-based suite to run against PostgreSQL with complete automation and reproducibility.

Practical uses

For server administrators, the main use case is twofold: comparing versions and patches of the same engine before a production upgrade, or comparing different engines on the same hardware with identical configuration. An example cited by Miller involves a regression on a simple select-by-id query: once verified with TAF, the fix delivered a performance gain over 3 times better in the micro benchmark tested. The source code, with sysbench and HammerDB suites already integrated, is available at github.com/MariaDB/TAF.