Mingxin Technology

How to Run Head‑to‑Head NVMe‑oF Throughput and TTFT Tests

Published 2026-09-08 · Mingxin Technology Insights

This guide explains how to design and run repeatable head‑to‑head NVMe‑over‑Fabrics (NVMe‑oF) comparisons focused on throughput and Time‑to‑First‑Token/Time‑to‑First‑Response (TTFT). It covers test planning, instrumentation, workload design, common pitfalls, and how to interpret results for AI inference and storage acceleration stacks.

Define the comparison scope and success criteria

Prepare a controlled environment

  1. Hardware parity: identical servers for initiators, same NICs, same firmware and driver versions. Lock BIOS/CPU P‑states and disable power management that changes performance across runs.
  2. NUMA and affinity: pin processes to CPU cores on the same NUMA node as the NIC and GPU; align memory allocation (hugepages where appropriate).
  3. Network: isolate test fabric to avoid background traffic. For RoCE test, ensure lossless fabric (DCBX/PFC) or use iWARP. Measure baseline packet loss and latency using ibbench or iPerf.
  4. Software stack: same kernel (or documented differences), NVMe target versions, initiator drivers, and IO schedulers. Record all versions for reproducibility.

Workload design for throughput and TTFT

Measurement and instrumentation

Test execution checklist

Interpreting throughput vs TTFT

Typical pitfalls and how to avoid them

Example comparison table: common NVMe‑oF options

Variant What it changes Typical trade-offs / measurement focus
NVMe/TCP Simpler stack; works over IP Easier to deploy; CPU overhead higher on initiator; measure kernel TCP CPU use and TX/RX queues
RDMA (RoCEv2/iWARP) Kernel bypass, lower CPU Lower latency and lower TTFT risk; requires lossless fabric or iWARP; measure P95/P99 latency and RDMA CQE rates
Storage acceleration (KV cache tiering) Adds a KV cache layer or local read cache Can greatly reduce TTFT by turning cold IOs into local cache hits; measure cache hit ratio, cache fill latency. Example: vendor‑provided signed benchmarks for an FX series 480B model report inference throughput +29–40% and TTFT −26–32% in production form (reports downloadable).

Reporting template (what to publish)

Key takeaways

If you want, I can draft a concrete test matrix and a fio jobfile plus an inference timestamping snippet tailored to your servers and the transport you plan to test.