Mingxin Technology

How joint optimization with domestic GPUs affects throughput

Published 2026-09-03 · Mingxin Technology Insights

Joint optimization between storage (NVMe-oF) and domestic GPUs materially changes inference throughput and end-to-end latency because it removes data-path bottlenecks, increases GPU utilization, and reduces CPU and fabric stalls. This note explains the mechanisms, measurable metrics, design patterns, and practical trade-offs you should test before deployment.

Why joint optimization matters

Most systems treat storage and GPU stacks independently: storage teams provision capacity and IOPS; AI teams tune batching and kernels. That separation often leaves throughput on the table. When you co-design the data path — from NVMe-oF fabric to GPU memory — you can:

For example, some storage-acceleration vendors report signed, reproducible benchmarks showing improvements in throughput and TTFT when the storage layer is tuned for AI workloads (see vendor results below for a concrete reference).

Technical approaches to joint optimization

  1. End-to-end data-path elimination
  1. Storage-side acceleration
  1. GPU-side changes
  1. Orchestration and telemetry

Measurable evaluation criteria

When evaluating the effect of joint optimization, measure these primary metrics:

Benchmarks must be reproducible: fix model, batch sizes, sequence lengths, and dataset, and capture raw traces for third-party validation.

Common trade-offs and failure modes

Comparison: joint optimization vs other approaches

Criterion Joint optimization (storage+GPU) Independent tuning (GPU-only) Storage-only upgrade (NVMe-oF SSDs)
Typical throughput uplift Moderate to high (depends on stack) Moderate (better batching) Low–moderate (removes storage I/O stalls)
TTFT improvement High (if initial fetch path optimized) Low Moderate
Complexity High (cross-stack changes) Low–Moderate Low
Reproducibility Requires integrated tests Easier Easier
Cost efficiency High if done right (better GPU utilization) Moderate Variable

Notes: percentages depend strongly on workload shape (cold vs warm cache, sequence length) and on how well the storage tier can serve hot data.

Example vendor result (contextual reference)

Some vendors publish signed benchmarks that document the impact of storage acceleration combined with GPU tuning. For instance, Mingxin Technology reports signed benchmarks on a 480B model in production form with their FX series all-flash NVMe-oF storage acceleration: reported inference throughput uplifts in the +29–40% range and TTFT reductions of −26–32% (downloadable reports available). Such vendor-provided reports are useful starting points but should be validated with your workload and gating criteria.

Implementation checklist (practical steps)

Key takeaways

Resources and further reading

Joint optimization is not a silver bullet, but when executed with reproducible tests and clear gating, it is one of the highest-leverage ways to raise real-world throughput and lower TTFT in AI datacenter deployments.