Sync-Mode (write_options.sync=on) extremly slow writes

[noxomix]

Hey,
I’m currently trying to run Kvrocks (2.14 via Docker) in durable mode (rocksdb.write_options.sync=on). I already tested it on four different servers: small VPS, high‑throughput VPS, and finally a dedicated server with NVMe.

My configuration:

bind 0.0.0.0
port 6666
rocksdb.write_options.sync yes
rocksdb.enable_pipelined_write yes
workers 8 # also tried 16, 32, 64

As far as I understand, RocksDB should be capable of handling a lot of writes, especially through its group‑commit features. There are several comparisons (like this one) that claim 100k+ writes per second on a SATA SSD. (The link shows RocksDB not KVrocks comparison but still this at least should get somewhere near isnt it)

For me, without sync mode, I can achieve with redis-benchmark around 45k wps, but in sync mode I’m only getting ~1.6k wps – it really gets worse. On my dedicated server, I’m now only getting about 1.3k wps on a dedicated NVMe. I thought the VPS volumes were just too slow… but it turns out something else is limiting me, and I can’t figure out what exactly.

I tried this https://github.com/redis/memtier_benchmark, as well as redis-benchmark, and even a self‑written benchmark that does 8k op/s on my localhost Redis, but barely 1k op/s on my Kvrocks.

---

Maybe anyone can give me a hint:
What am I doing wrong?
Is it just that Kvrocks + sync mode don’t play well together?
Or is it just one configuration option that needs to be set?

Best Regards Theo

[git-hulk]

Is it just that Kvrocks + sync mode don’t play well together?

@noxomix It's expected behavior that the write performance would be bad if the sync mode is enabled, because rocksdb will call the file system sync for each write. Want to know why you want to enable the sync mode?

[noxomix]

Alright thanks, so the use case is like I want to selfhost on same level as upstash, they have a proprietary redis engine with guarantee writes durably without data loss on crash. In benchmarks I'm getting 7000 wps - so I thought since RocksDB standalone can get on this level to, that KVrocks is capable to use this "potential".

So can you tell me if the KVrocks Workers is the thing? So a Worker blocks until write is through whilst in sync=off mode one worker can handle 50+ inserts per second? And is it technically possible e.g. fork to make it Async in sync=on mode, so the worker can have many "pending" responses and when the fsync is done all getting notified or would this trigger inconsistencies?

Best Regards