The stress test
Turso started with a workload that stress-tests their database to ensure data integrity. The test script:- Randomly generates a schema and table.
- Generates a specified number of random SQL insert, update, and delete queries.
- Executes the generated SQL queries.
- Periodically checks the database for integrity violations.
| Component | Code lines |
|---|---|
| Schema and table creation | gen_schema |
| Insert query | generate_insert |
| Update query | generate_update |
| Delete query | generate_delete |
| Integrity check | integritycheck |
Turso’s script uses our Rust SDK to provide randomness. While many programming languages and frameworks support built-in PRNGs, those initialize at runtime, which means you can’t regenerate a new random sequence without reseeding. This makes systematic exploration difficult, so Antithesis includes a few ways of providing randomness, either using our testing environment or our SDKs.
Running in Antithesis
Initially, Turso simply ran the stress test binary as an entrypoint in their first Antithesis testing setup, by including a single line of code in their dockerfile. That’s all you need to do to start testing with Antithesis! An equivalent approach, demonstrated below, is to use Test Composer to execute the stress test binary. To do this, Turso made their stress test a singleton driver command by placing it in a simple shell script. singleton_driver_stress.shTest Composer is an opinionated framework, so take note of the naming conventions and paths used. More details are here.
Containerization
Whichever approach you use, Turso’s containerization setup is a helpful example of how to set up your software in Antithesis.- Dockerfile (this is an updated version of the Dockerfile above)
- Orchestration (docker-compose)
- Ready signal
Test quality
Now Turso’s test is running in Antithesis’ testing environment, which means it’s running many thousands of times while being subjected to faults. Each test run now consists of:- Multiple test branches experiencing different faults.
- The software undertaking the same fixed sequence of operations on every branch.
Strengthening with Test Composer
The next step in the evolution is to break down the monolithic stress test into one that allows Antithesis to control execution order, parallelism, and frequency, and guide the fuzzer to more interesting system states. Here’s how the core components of the stress test map to Test Composer commands:
Apart from breaking down the monolithic stress test, some of the panics in the monolith are converted into always assertions. If the assertion fails or a panic is triggered, it’ll now show up as a property failure in the triage report. This allows Turso to see what the consequences of a given error are.
Containerization
This uses the same containerization and orchestration files as above, but look for the relevant lines that refer to this directory. Note that Turso’s current testing setup for Antithesis includes both thesingleton_driver_stress, and the collection of Test Composer commands.
Test quality
Each test run now consists of:- Multiple test branches with different sequences of events.
- Parallel insert/update/delete/rollback operations.
Summary
By following this progression, you move from writing fixed examples to defining properties and behaviors, letting Antithesis explore execution paths on its own.| Setup | What you’re testing | Will you find the bug? |
|---|---|---|
| Example-based testing, manual runs | Fixed test path | Only if it’s on the one path that you thought to test. |
| Example-based testing, partially randomized, manual runs | Test a different path per test run, one path per click | If randomization finds the buggy input sequence. |
| Example-based testing, partially randomized, automated runs | Test a different path per test run, many paths per click | If randomization finds the buggy input sequence. |
| Autonomous testing, singleton | Many executions with randomness + fault injection | If at least one combination of events and faults across the branches of execution leads to a bug. |
| Autonomous testing, composed | Fully autonomous, parallel, property-based testing | Very very likely! |