rustfs vs minio

[drivebyer]

Why is RustFS slower than MinIO in a 4 KiB benchmark comparison? Could my RustFS server be misconfigured?

rustfs：

minio：

rust yaml:

apiVersion: rustfs.com/v1alpha1
kind: Tenant
metadata:
  name: bench-rustfs
  namespace: bench
spec:
  image: rustfs/rustfs:latest
  imagePullPolicy: IfNotPresent
  podManagementPolicy: Parallel
  pools:
    - name: pool-0
      servers: 4
      persistence:
        volumesPerServer: 1
        volumeClaimTemplate:
          accessModes: ["ReadWriteOnce"]
          resources:
            requests:
              storage: 1Gi
      resources:
        limits:
          cpu: "2"
          memory: 4Gi
        requests:
          cpu: "1"
          memory: 1Gi
  env:
    - name: RUST_LOG
      value: "debug"

minio yaml:

apiVersion: minio.min.io/v2
kind: Tenant
metadata:
  annotations:
    prometheus.io/path: /minio/v2/metrics/cluster
    prometheus.io/port: "9000"
    prometheus.io/scrape: "true"
  labels:
    app: minio
  name: bench-minio
  namespace: bench
spec:
  certConfig: {}
  configuration:
    name: bench-minio-env-configuration
  env:
    - name: MINIO_PROMETHEUS_AUTH_TYPE
      value: public
  exposeServices: {}
  features:
    domains: {}
  image: quay.m.daocloud.io/minio/minio:RELEASE.2025-04-08T15-41-24Z
  imagePullSecret:
    name: ""
  mountPath: /export
  podManagementPolicy: Parallel
  pools:
    - affinity: {}
      name: pool-0
      resources:
        limits:
          cpu: "2"
          memory: 4Gi
        requests:
          cpu: "1"
          memory: 1Gi
      securityContext:
        fsGroup: 1000
        fsGroupChangePolicy: OnRootMismatch
        runAsGroup: 1000
        runAsNonRoot: false
        runAsUser: 0
      servers: 4
      volumeClaimTemplate:
        metadata: {}
        spec:
          accessModes:
            - ReadWriteOnce
          resources:
            limits:
              storage: 1Gi
            requests:
              storage: 1Gi
          storageClassName: local-path
        status: {}
      volumesPerServer: 1
  requestAutoCert: false
  sideCars:
    resources:
      limits:
        cpu: 200m
        memory: 200Mi
      requests:
        cpu: 100m
        memory: 100Mi

[drivebyer]

FYI @loverustfs

[loverustfs]

We conducted the test on a server with 2 cores and 4GB of RAM. The results are as follows:

MinIO

warp mixed --host=127.0.0.1:9000 --access-key=minioadmin --secret-key=minioadmin --obj.size=4KiB --duration=3m
╭─────────────────────────────────╮
│ WARP S3 Benchmark Tool by MinIO │
╰─────────────────────────────────╯

Benchmarking: Press 'q' to abort benchmark and print partial results...

 λ █████████████████████████████████████████████████████████████████████████ 100%

Reqs: 345967, Errs:0, Objs:345967, Bytes: 810.9MiB
 -    DELETE Average: 193 Obj/s; Current 198 Obj/s, 12.9 ms/req
 -       GET Average: 868 Obj/s, 3.4MiB/s; Current 869 Obj/s, 3.4MiB/s, 10.2 ms/req, TTFB: 10.1ms
 -       PUT Average: 289 Obj/s, 1157.8KiB/s; Current 278 Obj/s, 1110.8KiB/s, 12.7 ms/req
 -      STAT Average: 579 Obj/s; Current 584 Obj/s, 9.9 ms/req

UI: program was killed: context canceled
Report: DELETE. Concurrency: 20. Ran: 2m57s
 * Average: 193.00 obj/s
 * Reqs: Avg: 12.4ms, 50%: 11.7ms, 90%: 21.3ms, 99%: 34.3ms, Fastest: 0.7ms, Slowest: 95.0ms, StdDev: 7.0ms

Throughput, split into 177 x 1s:
 * Fastest: 208.43 obj/s
 * 50% Median: 193.93 obj/s
 * Slowest: 161.33 obj/s

──────────────────────────────────

Report: GET. Concurrency: 20. Ran: 2m57s
 * Average: 3.39 MiB/s, 868.34 obj/s
 * Reqs: Avg: 9.9ms, 50%: 8.9ms, 90%: 17.8ms, 99%: 28.7ms, Fastest: 0.6ms, Slowest: 93.6ms, StdDev: 6.1ms
 * TTFB: Avg: 10ms, Best: 1ms, 25th: 5ms, Median: 9ms, 75th: 13ms, 90th: 18ms, 99th: 29ms, Worst: 90ms StdDev: 6ms

Throughput, split into 177 x 1s:
 * Fastest: 3.6MiB/s, 933.97 obj/s
 * 50% Median: 3.4MiB/s, 869.67 obj/s
 * Slowest: 3.0MiB/s, 767.24 obj/s

──────────────────────────────────

Report: PUT. Concurrency: 20. Ran: 2m57s
 * Average: 1.13 MiB/s, 289.46 obj/s
 * Reqs: Avg: 12.5ms, 50%: 11.5ms, 90%: 20.9ms, 99%: 32.9ms, Fastest: 1.6ms, Slowest: 84.2ms, StdDev: 6.5ms

Throughput, split into 177 x 1s:
 * Fastest: 1252.9KiB/s, 313.22 obj/s
 * 50% Median: 1163.1KiB/s, 290.78 obj/s
 * Slowest: 1016.1KiB/s, 254.04 obj/s

──────────────────────────────────

Report: STAT. Concurrency: 20. Ran: 2m57s
 * Average: 578.83 obj/s
 * Reqs: Avg: 9.5ms, 50%: 8.5ms, 90%: 17.3ms, 99%: 27.7ms, Fastest: 0.4ms, Slowest: 97.2ms, StdDev: 6.0ms

Throughput, split into 177 x 1s:
 * Fastest: 630.43 obj/s
 * 50% Median: 582.23 obj/s
 * Slowest: 506.56 obj/s


──────────────────────────────────

Report: Total. Concurrency: 20. Ran: 2m57s
 * Average: 4.52 MiB/s, 1929.65 obj/s

Throughput, split into 177 x 1s:
 * Fastest: 4.9MiB/s, 2075.97 obj/s
 * 50% Median: 4.5MiB/s, 1934.75 obj/s
 * Slowest: 4.1MiB/s, 1707.83 obj/s


Cleanup

RustFS

warp mixed --host=127.0.0.1:9000 --access-key=rustfsadmin --secret-key=rustfsadmin --obj.size=4KiB --duration=10m
╭─────────────────────────────────╮
│ WARP S3 Benchmark Tool by MinIO │
╰─────────────────────────────────╯

Benchmarking: Press 'q' to abort benchmark and print partial results...

 λ █████████████████████████████████████████████████████████████████████████ 100%

Reqs: 1928655, Errs:0, Objs:1928655, Bytes: 4520.3MiB
 -    DELETE Average: 322 Obj/s; Current 226 Obj/s, 9.2 ms/req
 -       GET Average: 1449 Obj/s, 5.7MiB/s; Current 1063 Obj/s, 4.2MiB/s, 5.8 ms/req, TTFB: 5.8ms
 -       PUT Average: 483 Obj/s, 1932.5KiB/s; Current 350 Obj/s, 1401.8KiB/s, 8.2 ms/req
 -      STAT Average: 966 Obj/s; Current 707 Obj/s, 5.7 ms/req

UI: program was killed: context canceled
Report: DELETE. Concurrency: 20. Ran: 9m57s
 * Average: 321.93 obj/s
 * Reqs: Avg: 10.2ms, 50%: 9.8ms, 90%: 14.9ms, 99%: 22.7ms, Fastest: 0.8ms, Slowest: 114.3ms, StdDev: 4.2ms

Throughput, split into 597 x 1s:
 * Fastest: 382.37 obj/s
 * 50% Median: 323.93 obj/s
 * Slowest: 172.07 obj/s

──────────────────────────────────

Report: GET. Concurrency: 20. Ran: 9m57s
 * Average: 5.66 MiB/s, 1448.72 obj/s
 * Reqs: Avg: 5.5ms, 50%: 5.1ms, 90%: 9.0ms, 99%: 14.8ms, Fastest: 0.2ms, Slowest: 120.1ms, StdDev: 3.3ms
 * TTFB: Avg: 5ms, Best: 0s, 25th: 3ms, Median: 5ms, 75th: 7ms, 90th: 9ms, 99th: 15ms, Worst: 114ms StdDev: 3ms

Throughput, split into 597 x 1s:
 * Fastest: 6.7MiB/s, 1716.78 obj/s
 * 50% Median: 5.7MiB/s, 1456.23 obj/s
 * Slowest: 3.0MiB/s, 760.90 obj/s

──────────────────────────────────

Report: PUT. Concurrency: 20. Ran: 9m57s
 * Average: 1.89 MiB/s, 482.90 obj/s
 * Reqs: Avg: 9.3ms, 50%: 8.9ms, 90%: 13.2ms, 99%: 20.5ms, Fastest: 1.0ms, Slowest: 121.3ms, StdDev: 3.8ms

Throughput, split into 597 x 1s:
 * Fastest: 2.2MiB/s, 573.65 obj/s
 * 50% Median: 1940.8KiB/s, 485.20 obj/s
 * Slowest: 978.9KiB/s, 244.73 obj/s

──────────────────────────────────

Report: STAT. Concurrency: 20. Ran: 9m57s
 * Average: 965.79 obj/s
 * Reqs: Avg: 6.2ms, 50%: 5.9ms, 90%: 9.0ms, 99%: 14.5ms, Fastest: 0.3ms, Slowest: 126.8ms, StdDev: 3.0ms

Throughput, split into 597 x 1s:
 * Fastest: 1142.05 obj/s
 * 50% Median: 971.77 obj/s
 * Slowest: 490.79 obj/s


──────────────────────────────────

Report: Total. Concurrency: 20. Ran: 9m57s
 * Average: 7.55 MiB/s, 3219.35 obj/s

Throughput, split into 597 x 1s:
 * Fastest: 8.9MiB/s, 3795.56 obj/s
 * 50% Median: 7.6MiB/s, 3238.74 obj/s
 * Slowest: 3.9MiB/s, 1668.95 obj/s


Cleanup
Cleanup Done

[loverustfs]

Speed ​​optimization involves many aspects, and without understanding your environment and configuration, we cannot provide SLA-level optimization.