AMD Local AI Box vs DGX Spark Decode Test

If you want to run a 284-billion-parameter mixture-of-experts model on hardware you own, the AMD local AI box argument is about memory placement rather than raw compute. A 32 GB card cannot hold the file, but it can hold the weights every token reads.

AMD Local AI Box vs DGX Spark: What the Claim Actually Says

An AMD local AI box from Lucebox reports 51.1 tokens per second decoding DeepSeek V4 Flash on its Zero 495 workstation, against 14.09 tokens per second on one Nvidia DGX Spark. Both numbers come from Lucebox's own blog benchmark, and the two machines ran different quantized model files, so the ratio is a vendor claim rather than an independent measurement.

The Zero 495 pairs an AMD Ryzen AI MAX+ 495 APU with a Radeon AI Pro R9700 card. The APU exposes a large memory pool rated at 273 GB/s, the same figure the transcript attributes to the Spark, while the R9700 has 32 GB at 640 GB/s. Lucebox's blog states the R9700 reads memory roughly 2.3 times faster than the Ryzen chip, which matches 640 divided by 273.

The point of the design is not that the card can hold the model. DeepSeek V4 Flash carries 284 billion weights and the compressed build is about 98.3 GB, far past 32 GB. The card holds the subset of weights that every token touches, and the APU holds the rest.

The comparison also mixes three variables: expert placement on the fast card, a custom ROCm/ROCmFPX quantization format, and speculative decoding with an 11.3 GB draft model. Any one of them could move decode speed, and the sources do not separate their contributions.

How MoE Sparsity Makes a 32 GB Card Useful

Mixture-of-experts sparsity is the reason a 32 GB card can help with a model three times its size. Only a fraction of the weights participate in any single token, so the card only needs the weights that participate often.

Lucebox's guide describes V4 Flash as carrying 256 routed experts plus one shared expert across 43 layers, with the router selecting six of the 256 experts per token. DeepSeek's model card puts 13 billion of the 284 billion weights active per token, about 4.6%. Attention, the dense layers and the shared expert run for every token, and Lucebox calls that group the dense path.

Routing is not uniform. A paper on a different model, Mixtral, observed one expert in the final layer reused 15 or more times within a single sequence, and it also found that reuse counts even out after more than 1,000 sequences. Hotness depends on the workload, which is why Lucebox profiles representative requests into a hotness file before serving.

Each expert sits on exactly one chip. Within a layer, attention and the router run on the R9700, six experts are selected, and the experts pinned to the card execute there while the experts living in the APU memory pool execute on the Ryzen chip. The two results meet on the R9700, and the sequence repeats across all 43 layers.

What Crosses the Link Between the Two Memory Pools

Weights never cross the interconnect between the APU pool and the R9700. Only intermediate state tensors move, which keeps the slow link from becoming the bottleneck.

The DeepSeek V4 guide that Lucebox cites describes a state tensor of 4 x 4,096 numbers per token crossing the link at each layer boundary. At 32-bit floats that is roughly 64 KB per crossing, and the transcript notes the 32-bit width is an assumption rather than a stated fact. Even at a pessimistic per-number width, kilobytes are trivial next to the gigabytes of weights a token reads.

That asymmetry shapes the whole design. Because the link rate counts as a cost in Lucebox's allocator, the software keeps weights stationary and ships small results instead. An even split between the two branches would leave the faster card idle, since it would finish its half first.

Why the Expert Split Is Deliberately Lopsided

The allocator places experts to equalize time, not memory, which produces a deliberately uneven split. The R9700 holds at most about a third of the file by size, yet those experts are the most frequently selected ones, so the card performs well over a third of the memory reads.

Lucebox's blog says only the most selected experts go to the R9700 and that the allocator adds the next hot expert only when doing so shortens the longer branch. No source states how many experts per layer end up on the card or what share of router picks they capture.

The table below separates what each component does in this design.

Reading the Three Throughput Figures Correctly

The published throughput numbers come from different tests and should not be read as one measurement. Lucebox's serving test used a 2K-token prompt, 128 output tokens, two warm-up runs, three measured runs and temperature zero, reporting a median of 51.1 tokens per second against 14.09 for one DGX Spark, or about 3.63x.

A context-matched sweep at 2K, 4K, 8K and 16K with prompt caches off produced speedups from 3.09x to 3.60x. Tech Times reports an average of 47.75 against 14.09, which is about 3.39x. The README's headline 86 tokens per second is a third figure measured at 2K context.

Single-chip baselines disagree as well. The README lists the Ryzen chip alone at 42 tokens per second at 8K context, while the Hugging Face page for the 98.29 GB build lists 18.1 to 22.3 tokens per second on that chip alone. The sources do not explain the gap, so the size of both the single-chip baseline and the gain from the R9700 is unclear.

Speculative decoding is present in the 86 tokens per second result. An 11.3 GB draft model proposes several next tokens and the main model checks them together, at a reported draft acceptance rate of 0.97. One weights pass can therefore validate several accepted tokens, which can push throughput above a simple memory-bandwidth ceiling.

What Independent Evidence Is Missing

No independent reviewer has tested production Lucebox hardware. Units were expected to ship in January 2027, and every throughput figure in the comparison comes from Lucebox's own testing team.

A community repository dated August 21, 2026 measured 44 to 47 tokens per second on a single Spark using a 3-bit checkpoint that retains 216 of 256 experts per layer and supports contexts up to 384K. That configuration differs from Lucebox's, so it is not a matched comparison, but it suggests 14.09 tokens per second reflects the Q2 build and its software rather than a Spark ceiling.

The Spark side of the comparison also carries a price change. Lucebox's blog tested a 128 GB Spark at $4,699, the earlier price, while Crypto Briefing reports that model at $6,950 and a new 64 GB Spark at $4,999. The 64 GB model cannot hold a 98 GB file, so it is not the machine that was measured.

Decode speed is also not the whole picture. The README lists prompt processing at 788 tokens per second at 2K, and one outlet reports the Spark reading prompts faster at roughly 1,077 tokens per second, a secondhand figure that has not been checked. The 3.4x headline covers how fast answers stream, not how fast a long prompt is ingested.

Copying the Idea With llama.cpp on One Machine

You do not need two chips to apply the placement idea. A community llama.cpp guide splits a mixture-of-experts model into always-active parts and routed experts, keeping attention, dense layers and the shared expert on the GPU while routed experts sit in system RAM.

1. Serve your model with llama-server and set the -ngl 999 flag so every layer is offloaded to the GPU.
2. Add --n-cpu-moe with a layer count to keep that many layers' expert weights in system RAM, or use the --override-tensor exps=CPU form to push all expert tensors to the CPU.
3. Change the count in small steps and watch VRAM usage against measured tokens per second, because the best split depends on your model and GPU.
4. Record a before and after number for your own workload, since no published gain exists for this configuration.

The home version differs from Lucebox in two ways. The CPU computes experts held in RAM, where Lucebox uses a second GPU, and experts are selected by layer rather than by a profiled hotness file. You get the placement principle without the second card or the offline profiling step.

Dense models gain little from any of this. Every weight is used on every token, so there is no rarely used tail to leave in slower memory. The transcript cites Tech Times for that reasoning, and no benchmark in the material tests a dense model on this box.

Limits, Formats and Features That Block the Split

Format support constrains where this technique applies. Experts stored in adaptive formats cannot be split, and only fixed-codebook ROCmFPX files support the split that Lucebox describes.

The guide also states that page concurrency refuses to run with draft models and some other features, so speculative decoding and concurrency are not available at the same time. The best placement depends on the routing profile of your own workload, which means a hotness file tuned for one traffic pattern may not transfer to another.

Lucebox's README documents a three-tier mode that adds an SSD to the R9700 and the Ryzen chip. It lists 25 to 26 tokens per second on code for V4.1 Flash, a different model from the V4 Flash used in the headline comparison, so that figure is not directly comparable.

FAQ

• Is the Lucebox Zero 495 faster than a DGX Spark? Lucebox reports 51.1 tokens per second against 14.09 on one DGX Spark in its own serving test, roughly 3.63x. The result comes from the vendor, the machines ran different quantized files, and no independent reviewer has tested shipping hardware, so treat the ratio as a maker's claim.

• How can a 32 GB card help run a 98 GB model? It cannot hold the file. In a mixture-of-experts model only a small share of weights activates per token, so the card holds the dense path and the most frequently routed experts while the long tail stays in system memory. Weights never cross the link; only small state tensors do.

• Can I do this on my own PC without a second GPU? Yes, in a simpler form. Serve a mixture-of-experts model with llama.cpp, offload all layers to the GPU, and keep expert weights in system RAM with the --n-cpu-moe or --override-tensor exps=CPU options. Measure your own tokens per second before and after, because no published gain covers this setup.

• Does this help a dense model? Very little. A dense model uses every weight on every token, so there is no rarely used expert tail to leave in slower memory. The material contains no benchmark of a dense model on this hardware.

• Why do the single-chip numbers disagree? The README lists the Ryzen chip alone at 42 tokens per second at 8K context, while the Hugging Face page for the 98.29 GB build lists 18.1 to 22.3 tokens per second on that chip alone. The sources do not say why the figures differ, so both the baseline and the card's contribution remain uncertain.

• Is the 86 tokens per second figure comparable to the 42 figure? No. The 86 was measured at 2K context with speculative decoding, and the 42 at 8K context. Different context lengths and different software features make it a mismatched pair, not a before-and-after.

• How much of the 3.4x comes from speculative decoding? No source separates the contributions. Expert placement, a custom quantization format and speculative decoding with a reported 0.97 acceptance rate all feed the result, and the ratio compares formats and software as well as hardware.

• What is the fairest test of the claim? Running the same file and the same software on both machines. Until someone does that, the comparison measures a configuration stack rather than a hardware generation.

• When can I buy one? The first 200 units were priced at $5,999 for the 128 GB model, with a 192 GB version $2,000 more, and delivery was expected in January 2027. Those are vendor terms from the announcement, not shipping units in reviewers' hands.

A CTA test

Reusable test paragraph for Skala blog tooling checks, unrelated to the analysis above.

If you have a technical breakdown sitting in a recorded video, Skala blog turns the transcript into a structured article you can edit and publish.

Source video