Two DGX Sparks: Memory Over Speed
Adding two DGX Sparks to the lab: what they are, which models fit on one or need both, and how two of them share a model.
The lab so far is the 4090 and a spare MacBook. Now it has two NVIDIA DGX Sparks, connected with a QSFP cable.
What a Spark is
A DGX Spark is a small box built around NVIDIA's GB10 chip. The CPU and the GPU share 128 GB of memory, like on the Mac. It also has a ConnectX-7 network card with a 200 Gb/s port, which is what the QSFP cable plugs into. I measured 196 Gb/s across it.
With the Sparks, the lab has three kinds of memory:
| Machine | Memory | Memory bandwidth |
|---|---|---|
| RTX 4090 | 24 GB | about 1,000 GB/s |
| M1 Max | 64 GB | 400 GB/s |
| DGX Spark | 128 GB | 273 GB/s |
| Two Sparks | 256 GB | 273 GB/s each |
Each step down the table holds more and reads slower. A Spark can hold models the 4090 cannot load at all, but it reads them at about a quarter of the speed.
One Spark or two
There are three common ways to run a model on this pair.
TP=1, one model per Spark. If a model fits in 128 GB, it runs on one box. The other Spark is free for a second model, or a second copy of the same one.
TP=2, tensor parallel. Every layer of the model is split in half, and each Spark holds one half of every layer. For each token, both boxes read their half of the weights at the same time, so together they read at up to 546 GB/s. The cost is that the two halves have to exchange results after every layer, many times per token. That traffic is what the 200 Gb/s link is for. This is how most people run a model across two Sparks.
PP=2, pipeline parallel. The model is cut by layers instead. The first Spark holds the first half of the layers, the second holds the rest, and the work passes across the cable once per step. It needs much less traffic than TP=2, but for one user each Spark waits while the other works, so it adds memory, not speed. It is mostly a fallback for when TP=2 does not work well.
What runs on them
Here are some of the open models available right now, with their approximate size at 4-bit:
| Model | Type | Weights at 4-bit | Runs on |
|---|---|---|---|
| Qwen3.5-35B-A3B | MoE, 3B active | about 20 GB | one Spark |
| Qwen3.5-27B | dense | about 15 GB | one Spark |
| gpt-oss-120b | MoE, 5B active | about 65 GB | one Spark |
| Qwen3.5-122B-A10B | MoE, 10B active | about 65 GB | one Spark |
| Qwen3.5-397B-A17B | MoE, 17B active | about 210 GB | two Sparks, TP=2 |
A 4090 can run only the first two of these.
Most of the list is mixture of experts (MoE), and that is not a coincidence. A MoE model is large in total, but each token only reads the experts it uses. That suits a machine with a lot of memory and modest bandwidth.
The bandwidth arithmetic from the 4090 post shows the difference. Qwen3.5-27B at 16-bit is about 54 GB, and it is dense, so every token reads all of it. At 273 GB/s that is about 0.2 seconds per token, or 5 tokens per second at best. Qwen3.5-122B-A10B is more than four times larger, but each token reads only about 5 GB of active weights at 4-bit. The same arithmetic gives an upper limit of about 50 tokens per second.
These are limits on paper, not measurements. But they show which models this machine is built for: big, sparse ones, and the biggest of those need both Sparks.