Researched and written in cooperation with Claude.ai. Sources and caveats are listed at the end.
Here are two ways to put 256 GB of accelerator memory on your floor.
The first is an NVIDIA DGX-1: eight Tesla V100s at 32 GB each, a petaFLOPS of FP16 tensor throughput, 7.2 TB/s of aggregate HBM2 bandwidth, and a list price of $149,000 when it shipped. Today it turns up on the secondary market for something like $15,000.
The second is two NVIDIA DGX Sparks: desk-side GB10 Grace Blackwell boxes, 128 GB of unified LPDDR5x each, joined by a single cable. Eight thousand dollars, new, under warranty.
Same capacity. The used option brings roughly thirteen times the memory bandwidth and four times the FP16 throughput for twice the money. That looks like an easy call — and it is, just not in the direction the spec sheet suggests.
Three things decide this comparison, and none of them appear in a row labelled “memory.”
| Metric | 2× DGX Spark | DGX-1 (8× V100) |
|---|---|---|
| Accelerator memory | 256 GB LPDDR5x, unified | 256 GB HBM2, 32 GB/GPU |
| Aggregate bandwidth | 546 GB/s | 7.2 TB/s — 13× |
| Per-device bandwidth | 273 GB/s | 900 GB/s |
| Dense BF16 / FP16 | ~250 TFLOPS | ~1,000 TFLOPS — 4× |
| FP8 | ~500 TFLOPS | Not supported |
| FP4 | ~1 PFLOPS dense | Not supported |
| FP64 | ~1 TFLOPS | ~62 TFLOPS — 60× |
| Device-to-device link | 25 GB/s (200 Gb/s RoCE) | 300 GB/s NVLink 2.0 |
| Power draw | ~340 W | ~3,500 W |
| CUDA architecture | sm_121, current | sm_70, dropped in CUDA 13 |
Spark FLOPS are derived from NVIDIA’s “1 PFLOP FP4” headline halved down the precision ladder; treat them as ballpark. Volta’s petaFLOPS is dense — Volta has no structured sparsity.
Capacity is meaningless without precision
Volta’s tensor cores speak FP16. That is the entire list — no BF16, no FP8, no FP4. Blackwell’s fifth-generation tensor cores speak all of them natively.
This is usually filed under performance, but it is really a capacity multiplier. A parameter stored at FP4 occupies a quarter of the bytes it occupies at FP16. Same memory, four times the model.
| Configuration | Pooled memory | Max parameters |
|---|---|---|
| 1× DGX Spark | 128 GB | ~217B at NVFP4 |
| DGX-1, all eight GPUs | 256 GB | ~107B at FP16 |
| 2× DGX Spark | 256 GB | ~435B at NVFP4 |
| 4× DGX Spark | 512 GB | ~870B at NVFP4 |
Assumes ~85% of memory available for weights after KV cache, activations, and per-device context overhead.
Read the first two rows against each other. A single Spark, with half the DGX-1’s memory, holds roughly twice the model. Capacity parity at 256 GB still loses, because precision is a 4× lever and Volta cannot pull it.
It compounds from there, because models increasingly ship in those formats. DeepSeek-V3 is natively FP8. gpt-oss ships as MXFP4. On a V100 you must upcast them before you can run them at all — so a natively-FP8 checkpoint arrives twice its own size, and an MXFP4 one arrives four times.
There is an escape hatch, and it is worth being honest about it: llama.cpp builds cleanly for sm_70, and 4-bit GGUF weights are just bytes, so a 405B model at Q4 (~203 GB) does technically fit. Two things spoil it. --split-mode layer walks the GPUs in sequence, so you get roughly single-GPU bandwidth rather than the aggregate 7.2 TB/s — landing near 4 tok/s, no better than the Spark pair. And bitsandbytes 4-bit requires compute capability 7.5+, so QLoRA — the standard route to fine-tuning a large model on modest hardware — does not run on Volta at all.
Bandwidth is the thing you actually feel
Having established that the Spark holds more, here is where it loses badly, and it deserves to be stated plainly rather than buried.
Token generation is memory-bound. Every token requires streaming the model’s weights out of memory, so decode speed is governed almost entirely by bandwidth divided by weight bytes — a number you can work out before you buy anything.
70 GB ÷ 273 GB/s = 0.26 s → 3.9 tok/s ceiling
Llama 3.1 70B at FP8 on one Spark. Measured single-stream: 2.7 tok/s. The envelope predicts reality to within the efficiency factor, which is exactly the point — nothing about this number is a surprise if you do the arithmetic first.
For scale: that is 546 GB/s of aggregate bandwidth on the Spark pair against 7,200 GB/s on the DGX-1. Adding the second Spark does not fix it, either. Tensor parallelism across a 25 GB/s link means an all-reduce every layer over a pipe twelve times slower than NVLink 2.0. The second node buys capacity, not latency.
Two caveats keep this from being a rout. Batch serving closes much of the gap — a Spark reaches roughly 695 tok/s aggregate across 256 concurrent streams, because at that point you are amortising each weight read across many tokens. And mixture-of-experts models sidestep the problem structurally: with few active parameters per token, a 671B MoE moves less data per token than a dense 70B.
The scaling ceilings invert
Ask how far each architecture goes and you get opposite answers depending on whether you mean “by design” or “in practice.”
The structural difference is the size of the unit you scale from. A DGX-1’s unit is an eight-GPU NVLink domain: 256 GB at 7.2 TB/s, coherent internally, with four EDR InfiniBand ports — 400 Gb/s per node, double a Spark’s — sited at the corners of the cube mesh for GPU-to-GPU RDMA. A Spark’s unit is one GPU. There is no fast intra-node domain, because there is only one accelerator in the box, so every parallelism dimension beyond a single GPU crosses the slow fabric.
By design, it is not close. NVIDIA built the DGX-1 for POD deployments — nine per rack, reference architectures tested to six PODs — and proved it at 124 nodes with SATURNV, a fat-tree cluster that hit 4.9 petaFLOPS peak. The Spark’s supported ceiling is four nodes via the Cluster Assistant, reaching eight with breakout cables. The DGX-1 scales roughly fifteen times further.
In practice, that ceiling is historical. Each node you add is 3.5 kW of out-of-support hardware needing an InfiniBand switch, a rack, 240 V service, and real cooling.
| Pooled memory | DGX-1 path | Spark path |
|---|---|---|
| 256 GB | 1 node · ~$15k · 3.5 kW | 2 nodes · $8k · 340 W · one cable |
| 512 GB | 2 nodes + IB switch · ~$35k · 7 kW | 4 nodes + switch · $16k · ~700 W |
| 1 TB | 4 nodes · ~$65k · 14 kW | 8 nodes · $32k · 1.4 kW |
At every tier the DGX-1 delivers roughly 13× the bandwidth for about 2× the money and 10× the power. If bandwidth is what you need and you already own a datacenter, that is genuinely good value per GB/s. If you do not, the 1 TB tier arrives at 14 kW — about $24,500 a year in electricity alone.
Both architectures have a point past which you should buy something else. For Sparks it is around four to eight nodes, after which four RTX PRO 6000 Blackwell cards in one chassis give you 384 GB at roughly six times the per-device bandwidth with no fabric in the critical path. For the DGX-1, in 2026, that point is one node.
Sticker price is the smallest line item
Run both machines for three years at $0.20/kWh and the bargain evaporates — before accounting for cooling, the rack, or the electrical work.
| Configuration | Capital | Electricity, 3 yr | Total |
|---|---|---|---|
| DGX-1 · 8× V100 | ~$15,000 | $11,000–18,000 | ~$26,000–33,000 |
| 2× DGX Spark | $8,000 | $1,000–2,000 | ~$9,000–10,000 |
Excludes cooling overhead, rack, and 240 V service.
The “cheap” machine costs roughly three times the expensive one, and the crossover happens inside the first eighteen months. Power alone on the DGX-1 exceeds its own purchase price.
But the money is not the strongest argument. This one is: hardware depreciates on a slope; platforms depreciate on a cliff.
CUDA 13 removed offline compilation and library support for Volta. TensorRT-LLM, TGI, and Triton have all dropped sm_70 from current containers, leaving vLLM as roughly the last major serving stack that still accepts it. FlashAttention-2 requires sm_80. A DGX-1 bought today is pinned to CUDA 12.x permanently, and the gap between what it can run and what everyone else is running widens every month. No amount of money closes it, because there is nothing to buy.
When the old box is still the right answer
None of this makes a DGX-1 worthless, and if one is already racked and paid for, the analysis changes considerably.
It remains about sixty times a Spark at FP64, so any classical HPC — CFD, quantum chemistry, climate — still belongs on it and nowhere near a Spark. It is a perfectly good CI machine and a fast dataset-preprocessing box. It handles FP16 fine-tuning well for anything that fits in 32 GB shards, provided you stay on CUDA 12.x and vLLM. And its single-node bandwidth genuinely beats a small Spark cluster for dense-model inference.
If power is free to you — a university lab, a colo with flat-rate power — a large part of the cost table simply evaporates and the calculus shifts back meaningfully.
The mistake was never using one. It is buying more of them, or building a roadmap on a platform that has already been dropped by its own compiler.
Five rules for cheap silicon
- Multiply capacity by precision before comparing gigabytes. A machine that speaks FP4 has four times the effective memory of one that speaks only FP16. The spec sheet will not do this multiplication for you.
- Predict decode speed on an envelope. Weight bytes divided by memory bandwidth gives you tokens per second to within an efficiency factor. If that number disappoints you on paper, it will disappoint you in the rack.
- Price the electricity before the hardware. Three years at 3.5 kW costs more than the machine drawing it. Any comparison that stops at purchase price is measuring the wrong thing.
- Check the compiler support matrix before the benchmarks. Benchmarks tell you what a machine does today. The CUDA deprecation notice tells you what it will be allowed to do in two years.
- Ask what the scaling unit is, not just how far it scales. Eight GPUs behind NVLink and one GPU behind Ethernet are different starting points, and the difference shows up in every parallelism decision you make afterward.
Caveats
- Used DGX-1 pricing is thinly traded; the ~$15,000 figure is a market read, not a quote.
- Aggregate bandwidth figures assume tensor or expert parallelism. Under pipeline parallelism both machines fall back to single-device bandwidth, which flatters neither.
- Spark FLOPS are derived from NVIDIA’s headline FP4 number rather than measured; the 2.7 tok/s decode figure is a third-party measurement.
- The eight-node Spark configuration is a community ceiling using breakout cables, not a validated NVIDIA reference architecture. Four nodes is the number to count on.
Sources
- NVIDIA DGX Spark product page · NVIDIA Sync Cluster Assistant · Multi-Spark via switch
- DGX-1 system architecture whitepaper · DGX-1 datasheet · SATURNV cluster
- TensorRT-LLM support matrix · bitsandbytes 4-bit requirements · llama.cpp multi-GPU docs
How this was made
This post was researched and written in cooperation with Claude.ai. Every figure traces to the sources listed above; the caveats mark where an estimate stands in for a measurement.