Key takeaways

  • Juniper QFX5240 and NVIDIA SN5600 are both 64-port, 800GbE-class switches. A port-speed table alone does not decide an AI fabric.
  • Juniper sells QFX5240 as a switch platform running Junos OS Evolved, with RoCEv2 controls and Apstra Data Center Director automation. Spectrum-X is a broader NVIDIA platform that combines Spectrum switches with supported NVIDIA network adapters and a validated software stack.
  • Juniper's published GPU-backend design uses QFX5240 leaves for an H100 cluster, but its server and fabric links run at 400GbE. That is a reference-design choice, not the QFX5240's port ceiling.
  • NVIDIA's current Spectrum-X documentation supports BlueField-3 SuperNIC, ConnectX-7 and ConnectX-8 families in different configurations. It does not say every host requires a BlueField-3 DPU.
  • The vendors publish different throughput and power conventions. Treat headline numbers as planning inputs, not an apples-to-apples benchmark.

Start with the comparison boundary

“QFX versus Spectrum-X” is not a perfectly symmetrical product comparison. The QFX5240 is a family of Ethernet switches. Spectrum-X is an Ethernet platform spanning Spectrum switches, host adapters, reference architectures and software configuration.

For a useful infrastructure comparison, this guide uses the 64-port QFX5240-64OD and NVIDIA SN5600 as the switching layer, then separately examines what each vendor documents above the switch. It does not assume that matching faceplate speeds produce matching collective-communication performance.

800GbE switching: similar headline, different documentation

Juniper lists the QFX5240-64OD with 64 OSFP ports at up to 800GbE. Its datasheet reports 51.2 Tbps unidirectional and 102.4 Tbps bidirectional system throughput, a Broadcom Tomahawk 5 switch chip and Junos OS Evolved. Breakout modes support 128 ports at 400GbE or 256 ports at 200GbE.

NVIDIA's SN5000 hardware manual lists the SN5600 with 64 OSFP ports at 800GbE and maximum throughput of 51.2 Tbps. It can break out to 128 ports at 400GbE or 256 ports at speeds up to 200GbE.

Those rows look nearly identical, but the vendors label throughput differently. Juniper explicitly publishes uni- and bidirectional figures; NVIDIA's overview labels one maximum-throughput figure. Buyers should normalize directionality and the exact traffic test before turning either number into a capacity claim.

There is another important distinction between capability and deployment. Juniper's GPU Backend Fabric reference design uses eight QFX5240-64OD leaves per stripe for its H100 cluster. Each H100 server connects to each leaf at 400GbE, and the leaf-to-spine links are also 400GbE. The switch can expose 800GbE ports, but this particular lab topology does not use 800GbE end to end.

What the Juniper design actually validates

The Juniper design is a concrete three-stage IP Clos arranged around NVIDIA's rail-optimized stripe architecture. It uses eBGP for route advertisement, with L2 VLAN server attachment and an IRB gateway on the leaves.

In the published lab calculation, each H100 stripe has two servers, eight leaf connections per server and 400Gbps per connection. That produces 6.4 Tbps of server-to-leaf bandwidth per stripe. Four spine nodes and the documented leaf-to-spine links provide more fabric-side capacity in that example.

This is useful evidence that QFX switches can form a RoCEv2 GPU back-end around NVIDIA systems. It is not evidence that every QFX deployment has the same subscription ratio, latency or job-completion time. Cable layout, GPU count, rail mapping and the number of active uplinks still determine the result.

At the product level, Juniper documents RoCEv2 plus Priority Flow Control, ECN and DCQCN on QFX5240. The datasheet also lists dynamic load balancing, a PFC watchdog, telemetry and Apstra Data Center Director automation. Those are the controls an operator can use to build and validate a loss-aware Ethernet fabric; none removes the need to tune buffers, queues and congestion thresholds for the actual cluster.

What Spectrum-X adds above the switch

NVIDIA describes Spectrum-X as an AI-optimized Ethernet platform for east-west GPU traffic. Its current documentation combines Spectrum switches with BlueField-3 SuperNIC, ConnectX-7 NIC and ConnectX-8 SuperNIC families, depending on the reference-architecture version and GPU platform.

That corrects a common shortcut: Spectrum-X is not simply “SN5600 plus a BlueField-3 DPU in every host.” The current compatibility table includes several adapter families, and NVIDIA calls the BlueField-3 component a SuperNIC in this context. The host-side choice changes with the target system and topology.

NVIDIA's AI-factory networking reference attributes adaptive, telemetry-driven congestion control, fine-grained load balancing and hardware scheduling to the integrated platform. Network Operator exposes validated Spectrum-X profiles and rail or plane configuration to Kubernetes. That tighter switch-to-host validation is Spectrum-X's clearest architectural distinction from comparing standalone switch specifications.

It should still be read as a vendor architecture, not an independent benchmark. NVIDIA's performance language does not prove that every Spectrum-X installation beats a well-engineered QFX fabric. Workload collectives, topology, scale and software versions determine what a buyer should test.

Software and operating model

QFX5240 runs Junos OS Evolved. Juniper positions Apstra Data Center Director for intent-based setup, ongoing verification and rail-optimized design automation. This keeps the switching and fabric-automation layers visible as Juniper products while allowing the operator to select server adapters that meet the design.

NVIDIA offers SN5600 ordering variants with ONIE or Cumulus Linux, while Spectrum-X adds host firmware profiles and Kubernetes-oriented configuration through Network Operator. It is inaccurate to describe one Cumulus release as a single operating system spanning every Spectrum-X switch and host adapter. The integration comes from validated component combinations and coordinated configuration, not from pretending all components run the same software image.

The operational choice is therefore less about “open” versus “closed” and more about ownership boundaries:

  • A Juniper design can suit a team that standardizes on Junos and Apstra and wants to qualify its chosen host adapters and cluster topology.
  • Spectrum-X can suit a team that wants NVIDIA's documented switch-to-host reference architecture and its configuration tooling alongside NVIDIA GPU systems.

Both approaches still require version control, telemetry, failure testing and a repeatable congestion-tuning process.

Power numbers need normalization

Juniper reports 613 W typical power for the QFX5240-64OD, measured at 25°C with direct-attach cables, 50% IMIX traffic and no transceiver power. NVIDIA's SN5600 manual reports 940 W typical with passive cables under its stated ATIS method.

The figures are not a clean efficiency ranking. The test methods and disclosure language differ, and optics can materially change system power. A procurement model should compare the exact SKU, airflow, populated optics, traffic load and redundant-power configuration under one measurement boundary.

At cluster scale, switch watts are only part of the equation. Port breakout, oversubscription and the number of tiers can change how many chassis and optics the fabric needs. A design with the lower individual-switch figure can still consume more total network power if it requires more devices.

Decision checklist

Before selecting either architecture, ask vendors to demonstrate the proposed topology rather than a generic product maximum:

  1. Which switch and host-adapter SKUs are in the bill of materials?
  2. Are server links actually 400GbE or 800GbE, and how are ports broken out?
  3. What is the subscription ratio for each rail under the full GPU count?
  4. Which RoCE congestion-control profile was validated with the target collective library?
  5. Which software and firmware versions form the supported combination?
  6. What power boundary includes switches, optics and host adapters?
  7. What happens to job completion time when one link, leaf or spine fails?

Bottom line

Juniper QFX5240 and NVIDIA SN5600 occupy the same 64-port, 800GbE switching class, so port speed is not the decisive difference. Juniper documents a Junos-and-Apstra switch platform and a concrete QFX-based A100/H100 GPU-backend topology. NVIDIA documents Spectrum-X as an integrated switch, adapter and software architecture with validated host-side combinations.

The credible buying decision comes from testing the exact bill of materials and topology. Compare usable bandwidth per rail, congestion behavior, failure recovery, software ownership and whole-fabric power—not a single 800GbE badge.

Sources

This article was researched and fact-checked against the following sources: