NVIDIA TwoTower Diffusion LLM Boosts Speed Without Losing Quality
NVIDIA may have solved the main trade-off of modern LLMs
When working with any LLM, you have to choose between speed and quality. Autoregressive models like GPT deliver high quality, but run slowly because they generate one token at a time.
Diffusion models do the opposite. They generate whole blocks in parallel and therefore run faster, but until now this speed has had to be paid for with quality.
This is exactly the problem NVIDIA researchers set out to solve.
They took a 30B model and split it into two parts to generate tokens in parallel rather than one by one.
That’s how Nemotron-Labs-TwoTower appeared — a diffusion language model from NVIDIA Research, adapted on the basis of Nemotron-3-Nano-30B-A3B.
But before we dig into how TwoTower works, it’s worth understanding why diffusion LLMs generally lag behind autoregressive ones.
A diffusion LLM starts with a block of masked tokens and reveals them over several passes. On each pass, it reads the entire already generated text and simultaneously predicts the masked tokens in the current block.
So the same network is simultaneously performing two completely different tasks. It must understand the existing context while also predicting tokens that don’t exist yet.
One set of weights ends up being aimed at two different tasks at once, so the network can’t fully specialize in either of them.
TwoTower solves this problem by not forcing one network to do both tasks. Instead, a pretrained autoregressive model is split into two towers.
1. Context Tower — a frozen part of the model that reads the entire already generated text exactly as the original model did. It is no longer trained, so the model’s intellectual capabilities are fully preserved.
2. Denoiser Tower — a trainable part of the model that generates each new block of tokens in parallel. To avoid losing context, it constantly uses cross-attention to the Context Tower.
The most interesting part is how these two towers interact. They are connected layer-by-layer: the fifth layer of the Denoiser Tower interacts with the fifth layer of the Context Tower, the sixth with the sixth, and so on.
Thanks to this, the Denoiser gains access to the entire multi-level representation of the base model’s context, not just its final hidden representation.
Results:
> 2.42× higher generation throughput
> 98.7% of the original model’s quality is retained
> The architecture is built on a hybrid 30B Mamba–Transformer MoE model
> Training used only about 2.1 trillion tokens — just a fraction of the 25 trillion tokens used in pretraining the original model
The last figure is especially important here. None of the components were trained from scratch.
This is an adaptation that can be added to an existing autoregressive model. It relatively cheaply turns slow one-token-at-a-time decoding into fast parallel block generation.
So for any latency-sensitive systems, this is an approach that is definitely worth studying. You no longer have to choose between a model you trust and the speed you need.
The weights, code, and training methodology are in open access.