NVMe/TCP exposes remote NVMe namespaces as block devices across standard Ethernet and TCP/IP. It lets compute and storage scale separately without RDMA NICs or a lossless fabric. But the real question is whether network delay and shared resources fit your workload.
NVMe/TCP is remote NVMe, not an NVMe VPS
NVMe/TCP is an NVMe over Fabrics transport. It carries NVMe commands to remote storage over TCP/IP. An NVMe-labeled VPS does not prove that NVMe/TCP is involved.
A Linux initiator opens a TCP session to an NVMe target. The target exports an NVMe subsystem with one or more namespaces. Linux sees each namespace as a remote block device, like /dev/nvme0n1.
The I/O path includes the application, Linux NVMe driver, TCP/IP stack, Ethernet network, target controller, namespace, and physical SSDs. NVMe commands stay in NVMe format. ISCSI carries SCSI commands instead.
A namespace does not always mean one SSD.
A namespace can map to one drive, a RAID group, or a software-defined storage cluster. Think of it as a labeled storage slice. The label does not reveal the physical disks behind it.
“NVMe VPS” usually describes the flash media below the virtual disk. It does not describe the transport or storage layout. A VPS may use local SSDs, a shared flash pool, SAN-backed volumes, or a cached network layer.
Ask whether storage is local, network-attached, replicated, rate-limited, and protected by multipathing. The label alone cannot confirm separate storage or stable tail latency.
Ask a provider for measurable answers: p99 read and write latency, per-volume IOPS limits, throughput caps, storage replication method, failover time, and whether a virtual dedicated server can use more than one path to the target.
In a VPS platform, the hypervisor gives each guest virtual CPUs, memory, and disks. The physical storage design drives much of the real result. A volume may use local host NVMe, replicated storage across nodes, or a remote NVMe pool.
Local disks can give lower storage latency. They can also tie recovery to one host. Replicated remote volumes can improve mobility and availability, but add network round-trip time and shared-resource contention.
The most common mistake is treating the NVMe label as a performance guarantee.
For this reason, judge a Linux NVMe initiator by measured IOPS, throughput, and p99 latency. Do not judge it by the NVMe label alone.
TCP is fast enough for some loads, not all
NVMe/TCP can deliver high-throughput remote storage. It cannot match local PCIe NVMe latency. Every I/O crosses a network and uses CPU at both ends.
Measure the bottleneck you actually have
Test random 4 KB read and write IOPS. Also test sequential throughput, median latency, p95 and p99 latency, queue depth, CPU use, retransmits, and errors. Run tests at low concurrency and at expected production parallelism.
A 10 GbE link has a practical payload ceiling near 1 GB/s. A 25 GbE link reaches roughly 2.5 to 3 GB/s. Protocol overhead, encryption, TCP windows, and target design reduce those figures.
Use mixed fio workloads, not only sequential benchmarks.
P99 latency shows how slow the worst one percent of operations become. Average reads of 0.8 ms can hide p99 reads of 15 ms during backups. Those pauses can become visible database stalls.
Jumbo frames can reduce packet work. They work only when MTU matches across every hop. Setting MTU 9000 on hosts fails if a switch or VLAN remains at 1500.
That mismatch can cause drops, fragmentation, or failed connections. When choosing a transport, its operating cost matters as much as speed.
Choose NVMe/TCP, iSCSI, RoCE, or fibre channel
Choose NVMe/TCP for remote NVMe on normal Ethernet. Choose iSCSI for broad compatibility. Choose RoCE or NVMe over Fibre Channel only when lower latency justifies specialized operations.
| Transport | Typical network need | Latency sensitivity | Operations burden | Best fit |
|---|
| NVMe/TCP | Standard Ethernet, TCP/IP | Network RTT and CPU matter | Moderate | Virtualization, Kubernetes, private cloud |
| iSCSI | Standard Ethernet, TCP/IP | Higher protocol overhead | Low to moderate | Legacy and mixed-vendor estates |
| NVMe/RoCE | RDMA NICs and controlled fabric | Very low when tuned correctly | High | Microsecond-sensitive all-flash clusters |
| NVMe/FC | Fibre Channel HBAs and switches | Low and predictable | High, specialized | Existing FC data centers |
NVMe/TCP is usually the practical choice for shared high-speed storage on standard Ethernet. It avoids RDMA fabric demands and keeps NVMe commands. It is not best for microsecond-sensitive workloads where a well-run RoCE fabric earns its added complexity. Choose it when low-millisecond latency, normal TCP tools, and tested recovery meet the application’s needs.
How the storage request travels
Linux app→NVMe driver→TCP/IP→Ethernet→NVMe target→SSD pool
Each hop can add delay. Link speed, congestion, CPU, and target queues shape p99 latency.
iSCSI still makes sense for older operating systems and established SAN tools. It also fits teams that already test CHAP, MPIO, and recovery. It is rational when 5 to 10 ms storage latency is acceptable.
RoCE moves data with lower CPU overhead than normal TCP paths. It needs controlled switches, congestion management, compatible NIC firmware, and staff who can diagnose loss. NVMe over Fibre Channel fits sites with redundant FC fabrics and mature storage operations.
The decision is operational, not just technical.
When comparing providers, request the same latency and failover data for each option. Compare documented volume limits, zones, snapshots, and uptime terms. Do not assume a named transport in public cloud storage.
Deploy and test NVMe/TCP on Linux
A Linux NVMe/TCP setup needs a reachable target and matching TCP port access. It also needs nvme-cli, nvmet, and failure tests before production.
A minimal Linux connection sequence
Install nvme-cli on the initiator. Load the TCP transport with modprobe nvme-tcp. A typical discovery command is nvme discover -t tcp -a 10.0.20.10 -s 4420.
Then connect with nvme connect -t tcp -n nqn.2014-08.org.nvmexpress:example -a 10.0.20.10 -s 4420. On the target, nvmetcli can save an nvmet setup. Restrict access by NQN and firewall TCP port 4420.
Verify controllers and namespaces with nvme list, nvme list-subsys, and nvme list-ctrl. These commands confirm discovery and connection. They do not prove safe recovery under failure.
A successful connection is only the first test.
Test failure, multipath, and recovery
NVMe/TCP is not the best priority for one server with local NVMe. It also may not fit low-I/O workloads that already perform well on ordinary block storage. Avoid it when an application cannot tolerate added network microseconds. For tightly managed ultra-low-latency clusters, RDMA can justify its extra complexity.
For production, use NICs and switches that sustain required bandwidth without persistent drops. Isolate storage traffic with dedicated VLANs, interfaces, or separate Ethernet fabrics where practical. Standard TCP NICs are enough, so RDMA support is not required.
Checksum, segmentation, and receive-side scaling offloads can reduce host CPU pressure. Validate them before relying on them. The initiator and target need a kernel with NVMe/TCP support.
The target also needs reliable backing media and enough CPU for peak queues. Keep MTU consistent across every hop. Use at least two independent paths for multipath designs.
A common failure case is one working path and one blocked path. The volume looks healthy until a switch, NIC, or cable fails. Confirm firewall rules allow TCP port 4420 on every planned storage path.
This works well in theory, but recovery behavior decides production safety. Test link loss, target restart, path return, and application recovery before placing data on the volume.
Common questions
What is NVMe over TCP?
NVMe over TCP carries NVMe storage commands over standard TCP/IP Ethernet to remote NVMe namespaces. It normally uses port 4420. It does not require RDMA-capable NICs.
Is NVMe/TCP faster than iSCSI?
NVMe/TCP can have less protocol overhead than iSCSI, but speed depends on p99 latency, target CPU, link speed, and workload. A lightly used application may see little difference.
Does NVMe/TCP require RDMA?
NVMe/TCP does not require RDMA, RoCE, iWARP, or lossless Ethernet. It uses normal TCP/IP behavior. That usually fits existing Ethernet operations more easily.
Is an NVMe VPS using NVMe/TCP?
An NVMe VPS may use NVMe/TCP, but its marketing label cannot prove that design. Ask about storage layout, latency limits, replication, and the network path.
How much latency does NVMe/TCP add?
NVMe/TCP adds network and TCP processing delay beyond the SSD response time. Same-rack paths may stay below 1 ms round trip. Cross-region paths can add tens of milliseconds.
What matters most:- NVMe/TCP is remote block storage, not local NVMe extended by a cable.
- Measure p99 latency, CPU use, queue depth, and failure recovery alongside IOPS.
- Use standard Ethernet for simpler operations, but test the full network path under load.
- Choose iSCSI, RoCE, or Fibre Channel only after matching transport to latency and operations needs.
Choose the storage path before the provider
Use NVMe/TCP when shared high-performance storage needs normal Ethernet and predictable local-network latency. Your team must also test multipath recovery.
Before comparing OVHcloud Public Cloud, a VPS-3 OVH plan, or another virtual dedicated server, collect the same facts. Ask about storage locality, p99 latency, sustained throughput, IOPS limits, redundancy, and failover behavior. Also compare dedicated server and VPS storage isolation.
Choose locations based on user and workload latency needs.
A VPS, cloud instance, and dedicated server can all use NVMe media. Each makes different trade-offs. A VPS shares a physical host and may have fixed IOPS or throughput quotas.
Those quotas can make steady performance depend on provider rules and neighbor activity. Cloud hosting often adds API-based provisioning, snapshots, replicas, and remote block volumes. That model fits elastic applications and private-cloud designs.
A dedicated server gives direct control of local drives, NICs, and operating-system tuning. It often gives the lowest local latency. The customer must design redundancy and replacement procedures.
NVMe/TCP matters most when compute and storage must scale independently. One dedicated machine may remain the better choice when it can safely keep all data locally.
Learn more
Here are some additional resources on this subject:
