What is speculative decoding in AngelSpec?

Speculative decoding in AngelSpec is a training workbench for six drafter architectures, not a faster runtime you install. Tencent's Apache-2.0 repository lets you train multi-token prediction, block-diffusion, EAGLE-3-style, and DSpark-style drafters against your own target model, then compare accepted length on your own traffic before committing to one.

Speculative decoding in AngelSpec: what the repository is

Speculative decoding in AngelSpec is a training workbench, not a runtime, so installing it makes nothing you already serve faster. You point it at a target model, generate hidden states, train a drafter against them, then serve the resulting drafter with your own inference stack.

Tencent published the repository under Apache-2.0 and shipped it with seven drafter model files. The repository is a PyTorch training pipeline: multi-token prediction (MTP), block diffusion, DFlash, DFlare, DSpark, and D-Fly all sit behind a single configuration flag, which is the practical reason to use it. You can train each architecture against the same target model and the same benchmark set instead of comparing numbers that came from six different papers.

The distinction matters because the two file types being compared in public discussion do different jobs. A released drafter checkpoint is a file you pull and run. The toolkit is the thing that produced it. Holding both to the same download counter is not a comparison.

Why autoregressive decoding is memory bound

Autoregressive decoding emits one token per forward pass, and each pass must read the model's weights from memory, so a large model moves hundreds of gigabytes across the bus per generated word. The arithmetic performed on those weights is trivial by comparison, which is what it means to be memory bound. Roofline analysis places this kind of decoding at roughly one floating-point operation per byte moved, deep in that regime.

Speculative decoding exploits the same trip. A small draft model guesses several tokens ahead; the target model then verifies every guess in a single forward pass and keeps the longest prefix it would have written itself. The accept-or-reject rule is constructed so surviving tokens follow exactly the target model's distribution, not an approximation of it, which is why the output is not a quality trade.

The unit that governs the speedup is accepted length: how many tokens survive each verification round. Every accepted token past the first is a memory trip you did not pay for, so accepted length is the raw material the throughput multiplier is built from. Vendor pricing for a B200 class GPU ran from $3.49 to $14.24 per hour across clouds in April 2026, and cost per token is that rate divided by the tokens you actually serve, which is why idle memory bandwidth is expensive.

Two drafting families: multi-token prediction and block diffusion

Multi-token prediction bolts a small extra head onto the model and predicts a few tokens ahead. It is light and stable, and Tencent's Hunyuan 3 ships with one: a 3.8-billion-parameter layer inside a 295-billion-parameter model whose job is guessing ahead. Block diffusion instead produces a whole block of tokens in one pass, resolving a block the way an image model resolves a picture out of noise.

The two families behave differently because of entropy. Chat traffic leaves dozens of plausible next tokens, so a long parallel guess mostly gets discarded. Code and mathematical continuations are far more predictable, so a long block lands more often. The claim at the center of the AngelSpec paper is that no single drafting structure performs best across real-world workloads, and the entropy difference is the mechanism behind it.

Tencent's response was to train specialists rather than one drafter on a blended mixture: MTP on conversational data, block diffusion on code and mathematics. That choice is the reason the repository exists in its current shape, because which specialist you need depends on your own traffic mix, not on a public average.

AnglSpec's six drafters as a lineage: EAGLE-3, DFlash, DFlare, DSpark, D-Fly

EAGLE-3, published at NeurIPS 2025, kept autoregressive guessing but fuses features from several layers of the target model, and Tencent reports it reached a claimed 3x to 6.5x speedup over ordinary decoding. DFlash, from a University of California, San Diego lab, moved to whole-block guessing and reported roughly 2.5x the speedup EAGLE-3 could reach; it went to ICML 2026, and its arXiv preprint is dated 2026.

DFlare then attacked DFlash's weak spot. DFlash handed every draft layer the same fused summary of the target model, leaving no layer able to specialize, so DFlare let each layer learn its own blend. DSpark, from Peking University and DeepSeek and dated 2026, attacked the other end: a parallel block has no token that knows what was chosen before it, so acceptance decays as the block grows, and DSpark put some sequential information back.

D-Fly, Tencent's own architecture from the AngelSpec paper, combines DFlash's shared projection, DFlare's per-layer refinement, and an autoregressive correction head. DSpark and D-Fly were published three weeks apart by separate organizations that reached the same conclusion independently: pure parallel drafting discards too much, so some sequence has to return.

Dates in the published record: EAGLE-3 in March 2025, DFlash in February 2026, DFlare in June 2026, DSpark and D-Fly alongside the AngelSpec paper in July 2026. Read in sequence, six names in one repository become about seventeen months of architecture work rather than a menu of interchangeable options.

One boundary to keep straight: EAGLE-3 was developed by researchers at Princeton and other institutions, while DFlash came from a UC San Diego lab led by Yejun Liu, who also works on efficiency at NVIDIA. The AngelSpec repository implements and trains against these designs; it did not originate them, and D-Fly is the architecture Tencent contributed.

Accepted length and throughput: which AngelSpec numbers to trust

On Hunyuan 3 (Hy3-A21B), the paper's table reports mean accepted length of 3.00 for multi-token prediction, 3.69 for DFlash, and 4.79 for D-Fly. The gap between 3.69 and 4.79 is the roughly 30 percent improvement the abstract leads with, and it concerns accepted tokens per verification round, not end-to-end speed.

On a second model the picture narrows. D-Fly reaches 5.41 accepted tokens against DSpark's 5.32, close enough to call a tie on that model. That table is the most useful page in the paper precisely because all six architectures were run against one model on one benchmark set, which few labs publish.

For end-to-end throughput, the paper's range for D-Fly is 1.98x to 2.40x over ordinary decoding, tested from 4 concurrent users up to 64. A widely circulated 2.86x figure comes from Alpha Signal's write-up describing a peak on code and mathematics specifically, and that number is not in the paper. Size hardware against the paper's range, and note that every one of these multipliers was measured on a different model, hardware configuration, and load than the next.

D-Cut: batch-level verification budgeting for live serving

D-Cut treats verification as one shared pool of compute across the whole batch instead of one request at a time. It scores each request by how confident the drafter was, prices that work against a profiled cost model of the actual hardware, then spends verification budget where it pays off: harder checks where the draft was confident, almost none where it was a coin flip.

On live Hunyuan 1 traffic at 64 concurrent requests, Tencent reports D-Cut delivering 15.7 percent more throughput than D-Fly alone, while mean accepted length slips from 2.50 to 2.46. That trade is what separates a training result from something that holds up under production concurrency, and it is the part of the work most likely to be adopted elsewhere first.

Why trained data can matter more than architecture

Hold the architecture fixed and change only the training data, and D-Fly's own ablation moves as much as an architecture swap would. Tencent's backbone change alone took accepted length from 3.77 to 4.40, an autoregressive head took it to 4.60, and switching the training data to code and mathematics took it to 4.75. On the same paper's reported comparison, mean acceptance rate rises from 52.8 percent to 66.4 percent when only the training changes.

That is the two-specialist argument proved on the authors' own table. A blended drafter is a compromise, and the compromise costs acceptance. It also explains why the multi-token prediction result on conversational traffic and the block-diffusion result on code traffic are not contradictory measurements.

Repository state, licensing, and the open bug in the DSpark head

The repository has three commits, the last landing on 31 July 2026. On 30 July 2026 one account filed ten issues within seconds, evidently automated, and the reports are specific: a background refill task that fails silently on any exception, a cleanup routine that can delete the best checkpoint while the pointer file still names it, and a loss path that flattens a tensor and breaks the DSpark head downstream.

That last one means one of the six advertised architectures has an open correctness bug in its training code. The same account closed most of its own reports minutes later, and three remain open. The pull request carrying the fixes is still unmerged more than three weeks on.

GitHub renders the license as "Other," which reads as proprietary, but the license file itself states Apache-2.0 with a Tencent preamble that automated detectors fail to parse. The repository is genuinely open source under Apache-2.0; the detection label is wrong.

When to use AngelSpec, and when a released drafter is better

For faster local generation this week, use a released drafter checkpoint rather than a training workbench. AngelSpec on its own makes nothing you already run faster: you must point it at a target model, generate hidden states, train a drafter against them, and serve the result, with training sequences reaching 128K split across GPUs using sequence parallelism. If you are not already running your own model on your own hardware, none of that applies.

For the smaller group serving one model on their own hardware at real concurrency, the toolkit is worth more than any single checkpoint because it is the only place where all six architectures can be trained against your own target model to find out which one your traffic wants. That is the practical reason to run it, and it is a different question from which drafter is fastest in a headline.

The comparison below is stated at the level the evidence supports, not as a full ranking.

FAQ

• What is speculative decoding in AngelSpec? It is a training workbench that trains a small drafter model to guess tokens ahead, which the large target model then verifies in one forward pass. AngelSpec unifies six drafter architectures behind one configuration flag and is published under Apache-2.0 by Tencent.

• Does AngelSpec make my inference faster right away? No. It is a training framework, so it produces a drafter you then serve. Without your own target model and hardware, the pipeline does not apply to you.

• What is accepted length and why does it matter? Accepted length is the average number of tokens that survive each verification round. Tencent's paper reports 4.79 for D-Fly on Hunyuan 3 against 3.69 for DFlash and 3.00 for multi-token prediction, and every accepted token past the first avoids a memory read.

• Is the 2.86x speedup figure real? It appears in a secondary write-up describing a peak on code and mathematics, not in the paper. The paper's end-to-end range for D-Fly is 1.98x to 2.40x over ordinary decoding, measured from 4 to 64 concurrent users.

• Is AngelSpec licensed as open source? Yes. The license file states Apache-2.0 with a Tencent preamble. GitHub's license detector prints "Other" because it cannot parse that preamble, which is a labeling failure, not a proprietary license.

• Is D-Fly better than DSpark? On Hunyuan 3 they are close: 5.41 accepted tokens against 5.32, effectively a tie. Which architecture wins depends on the model and the traffic mix, which is the paper's central claim.

• Does D-Cut improve throughput in production? Tencent reports 15.7 percent more throughput than D-Fly alone on live Hunyuan 1 traffic at 64 concurrent requests, with mean accepted length slipping from 2.50 to 2.46. That is a vendor-reported measurement on one workload.

• Are there known bugs in the training code? Three issues remain open, including a loss path that flattens a tensor and breaks the DSpark head, which affects one of the six architectures. The pull request with fixes was still unmerged as of September 2026.

• Should I use a released drafter or the toolkit? Use a released checkpoint, such as one with an MIT license and ready backends for MLX on Apple hardware and OpenAI-compatible servers, if you want speed now. Use AuthorSpec's training pipeline when you serve your own model at concurrency and need to test which drafter suits your traffic.

Source video