The goal of this program is to automate basic correctness checks and stress tests.
To run:
$ cd cmd/stress
$ go run main.go util.goWith the go-deadlock package turned off and the default sync
package on, I get throughput of around 20k-40k entries/second with
this stress test.
This program runs a few servers in memory but still communicates over localhost and actually writes/reads from disk.
The state machine tested is still a key-value state machine.
Most of the settings are configurable by editing ./main.go. But it's not particularly clean code.
It attempts to do the following:
- Start three servers and wait for a leader to be elected.
- Insert
N_ENTRIESinBATCH_SIZEacrossN_CLIENTS. - Validate that all servers have committed all messages they are aware of.
- Validate that all servers have all entries in their log in the correct order that entries were inserted.
- Shut down all servers and turn them back on. Validate that a leader has been elected.
- Validate that all messages that were inserted before shutdown are committed and in the log in the correct order.
- Shut down all servers and delete the log for one server.
- Turn all servers back on.
- Validate that a leader has been elected.
- Ensure that all servers have all entries (i.e. that the deleted log has been recovered).
That is: test the basics of leader election and log replication.
One variation that I run manually at the moment is to have three servers configured but only turn on two of them and ensure the entire process still works (ignoring testing for entries on the down server). This is to prove that quorum consensus works for leader election and log replication.