Next-Concept Modeling: From Sketch to Clear Text

“However, it is a important to the the the and the.” That is what a translation looks like after a model has finished the first of eleven passes. After the eleventh it reads “However, there is concern that small and medium-sized enterprises will remain vulnerable to hacking and surveillance.” The sentence comes from the paper on Next-Concept Modeling (NCM), which a team from Stanford and Together AI posted on arXiv on October 9, 2026. The idea is borrowed from image generation: first a rough sketch of the whole text, then sharpening, everywhere at once.

One Token After Another

Almost every language model we work with today is autoregressive. It predicts the next token, appends it to the text and then computes again. A sentence of twelve tokens needs twelve passes through the network, and each one waits for the previous one. Reasoning models make this chain longer because they also write their thinking in tokens. OpenAI states in its reasoning guide that these tokens are not visible via the API, yet still occupy space in the context window and are billed as output.

The chain is the core problem. Speculative decoding and multi-token prediction soften it by guessing several tokens and checking them in one pass (what that looks like locally is covered in the MTP test on the M3 Max). They do not change the principle. The text is produced from left to right, and a token, once written, stays.

Diffusion as the Previous Alternative

The best-known alternative is diffusion language models. They start with a sequence of masked or noised positions and denoise all positions in several steps at the same time. This has long since left pure research. The NCM paper names LLaDA as an open model with billions of parameters, and Mercury and DiffusionGemma as systems in production.

The authors see the following weaknesses in these models, which NCM is meant to fix:

1. Many steps. Reducing the number of denoising steps quickly costs quality.
2. Expensive steps. Every step processes the full sequence length, and bidirectional attention prevents the KV cache, the buffer for positions already computed, from being reused the way autoregressive models do.
3. Large training budget. Strong baselines such as MDLM and DUO run for one million steps on OpenWebText and see more than 500 billion tokens in the process.

The speed comparisons in the paper are built accordingly. NCM competes against diffusion and flow models of similar size. Flow models are closely related to diffusion; they learn a continuous path from noise to text. Autoregressive models appear only in the quality comparisons, not in the runtime measurements.

Text as a Stack of Corrections

NCM works in two stages. The first is a tokenizer of its own, which does not split text into word pieces but into a hierarchy of discrete codes. Technically this is a multiscale residual VQ-VAE. The name sounds harder than the idea is.

An encoder turns every token into a vector, a point in space, as with embeddings. Vector quantization replaces each of these vectors with the nearest entry from a fixed dictionary of vectors, the codebook. After that, the entry’s index is enough.

This is lossy; a residual remains. Residual quantization takes exactly this residual and quantizes it with a second codebook, then the residual of that with a third, and so on. Each level stores only what the levels above it got wrong. The method comes from image and audio codecs.

NCM combines this with different lengths. The top level consists of only a few codes for the whole text, every lower level is longer, and the bottom one has exactly one code per token. Before a level is quantized, the residual is scaled down to its length, then stretched back to full text length and added to the reconstruction so far. The sum of all levels yields the text. The upper levels hold what the authors call a “concept”, the coarse information about the whole passage. The name has only a loose connection to concepts in the sense of notions or ideas.

To make sure the coarse levels actually store something useful, the authors train with nested dropout. During training, the hierarchy is randomly cut off after the k-th level, and the decoder has to reconstruct the text from this partial stack. The upper levels therefore cannot rely on the lower ones to do the work.

The reconstruction is almost lossless, measured as exactly recovered tokens on held-out data (authors’ figures):

The authors point out themselves why this matters more than with images. Nobody notices a wrong pixel, but a wrong token can change the meaning of a sentence or invalidate a Sudoku solution. The errors observed are mostly mix-ups of related words or names.

One Level per Pass

The second stage is the actual generator, a transformer trained with the tokenizer frozen. It generates the hierarchy from top to bottom. In each step it sees the reconstruction so far from all coarser levels and predicts all codes of the next level at once. With K levels, a text needs K passes, no matter how many tokens it has. The first passes are also cheap because the coarse levels are short. At the end, the tokenizer’s decoder translates the complete reconstruction back into tokens.

The model it follows is VAR (Tian et al., 2024), an image model that first generates a picture at very low resolution and then finer, stage by stage. Text, however, lacks a natural counterpart to downscaling an image. What “coarse” means for a sentence is something the tokenizer has to learn first.

Against errors that build up during generation, the authors use another trick. In training, the generator always sees the correct coarser levels; during generation, only its own, possibly wrong predictions. The authors therefore deliberately replace part of the input codes during training with other codes from the same codebook, more often for coarse levels than for fine ones. The targets stay correct. According to the ablation experiments, in which one component is changed at a time, moderate corruption improves quality significantly, while too much of it hurts.

The translation trajectory from the paper’s appendix shows what this looks like during generation. The BLEU scores measure agreement with the reference translation:

The intermediate states do not improve steadily; pass 9 is worse than pass 8. The paper notes this explicitly. My reading: each level corrects the sum of the previous ones, and a correction can temporarily worsen something that is right in the end. The paper also notes occasional artifacts at the transitions between coarse levels.

The Numbers Against Diffusion Models

The main models have about 100 million parameters. Runtime was measured for 1,024 sequences at batch size 16 on a single Nvidia H100 SXM. All numbers are the authors’ own, from a preprint without peer review.

NFE stands for number of function evaluations, the number of passes through the network. The speed comes from fewer and from cheaper steps. ROUGE-L measures how well a summary matches the reference. On XSum, NCM without further tricks needs 8 passes for a ROUGE-L of 28.0; AURORA-LM needs 128 for 26.6.

The time saved can be turned back into quality. With minimum Bayes risk decoding, NCM generates several candidates and picks the one that is most similar to the others on average. With 16 candidates, NCM reaches a ROUGE-L of 29.6 on XSum at 128 passes, ahead of AURORA-LM with guidance (28.6 at 184 passes).

NCM also saves on training. On LM1B, tokenizer and generator together process 13.36 billion token positions, the diffusion baselines 66 billion, about 4.9 times more. The authors suspect the reason is that a fixed coarse-to-fine order is easier to learn than the many possible orders of a diffusion model. This is phrased as a hypothesis, not a finding.

In translation, NCM falls behind:

Without additional methods, NCM loses to a small autoregressive model and to the best diffusion model, but needs only a sixth of the passes. Anyone who wants the quality of the autoregressive model ends up, with MBR, at more passes than that model needs.

Sudoku as a Stress Test

The most striking result does not concern language. In Sudoku every digit has to satisfy row, column and box at the same time, and a single wrong digit makes the solution worthless. The models here are tiny, around 25 million parameters. Difficulty is set by the number of given cells (40, 35 and 30).

The autoregressive model solves one percent of the hard puzzles, NCM 86 percent. The authors’ explanation is plausible. A model that writes from left to right commits to each digit before it knows the rest of the board and cannot take it back. NCM works on the whole board at every level and can correct earlier partial solutions. In the example from the appendix, only 15 to 16 of the 81 cells of the decoded board match the solution after the first three steps, counting the given cells. After the twelfth, all of them do.

The gap to the diffusion and flow models, however, is much smaller than to the autoregressive model. FMLM+ solves 81 percent of the hard boards. The Sudoku result speaks mainly against strict left-to-right generation for tasks with global dependencies. It does not show that NCM is far ahead of all parallel methods.

How Practical Is NCM Today

NCM develops parallel text generation further and is therefore still at the beginning. The paper names most of the limits itself:

• Size. The main models have around 100 million parameters, the largest one tested 725 million. Current chat models are orders of magnitude larger. Whether the advantages grow, shrink or stay with size is open. The authors themselves call the scaling results up to 725M preliminary.
• Fixed shape. The number of levels and the length of each level are fixed. Each dataset has a fixed target length, and even a short continuation runs through the whole hierarchy. For a chat whose answer is sometimes one sentence and sometimes three pages, an essential building block is missing.
• One model per task. A separate tokenizer and a separate generator were trained for each dataset. There is no general model that translates, summarizes and solves puzzles.
• Two separate training stages. The tokenizer is learned independently of the generator. The authors name joint training as a possible improvement.
• Comparison class. All speed comparisons are against diffusion and flow models. There is no runtime measurement against an optimized autoregressive model with a KV cache.

As one direction, the authors name a schedule in which the number and length of levels depend on the input, so that simple texts cost less compute than difficult ones. Only a building block like that could make NCM usable for open-ended text lengths.

An Agent That Thinks Without Words

Four days after NCM, on October 13, 2026, a second preprint appeared, again with Stanford and Together AI, plus UIUC and Meta. Hybrid Reasoning Agents (project lead according to the paper: Jiaru Zou) does not start with writing but with thinking. The two works share no technique, but they target the same habit of spelling everything out in tokens.

A HybridAgent works in two modes. In text mode it behaves like a normal language model. When it writes the special token <latent>, it switches to latent mode.

There, no token is chosen anymore. The last hidden state, the vector the model computed after its final layer, passes through a small adapter called HybridLink and is used directly as the next input. The adapter is a residual block of two linear layers with about 10 million parameters for a 4B model. It translates the hidden state back into the space of input embeddings.

After every silent step, the model checks which token in its vocabulary is closest to the current vector. If it is </latent>, it returns to text mode, for example to issue a web search or write the answer. Across multiple turns, the entire trajectory of text, silent steps and tool results stays in the KV cache, which the authors call working memory. Standard inference servers cannot handle this switching, so the team extended its own infrastructure.

Training was done on Qwen3-Base with 4, 8 and 14 billion parameters. The 54,000 training trajectories were created automatically from existing text agent trajectories. External language models and the target model’s uncertainty decided which reasoning steps become silent and which stay as text. Tool calls and steps important for review stayed text. In the subsequent reinforcement learning there were rewards for correct answers, correct format and tool use, but none for silent thinking itself. The agent learned on its own when to think silently.

The results on the agent benchmark GAIA are clear (authors’ figures, overall score in each case):

On Humanity’s Last Exam (HLE), several frontier models score almost twice as high. Compared with text-only agents trained the same way, the authors report 35.5 to 54.7 percent fewer tokens, 1.8 to 2.6 times faster inference and 2.7 to 3.7 times lower cost. The cost is an estimate from compute per question, assuming 2.50 dollars per H100 hour and 10 percent utilization. It is not an API bill.

The authors test with interventions whether the silent steps actually mean something. If the silent steps of the 14B model are replaced with those from a different problem, the accuracy on MATH500 drops from 88.2 to 48.6 percent. If each silent vector is replaced with the nearest real token, it only drops to 87.2 percent. The silent thoughts therefore lie close to words, and that is exactly what makes them readable. Via the nearest token, every silent step can be output as bumpy pseudo-text. The repository shows an example: “Wait, I me the the speed of speed in air at 20° is about 343 meters per second. But I should verify that.”

Silent Thoughts, Readable Only by the Operator

At this point the HybridAgents paper turns political. The authors write that frontier labs such as OpenAI and Anthropic withhold the chain of thought of their reasoning models to protect intellectual property. With HybridAgents, they argue, the silent thoughts exist only as hidden states, readable solely through the provider’s HybridLink. So there would be no text that could leak. At the same time, the provider could check the full readout for safety problems.

For a provider, that is a business argument. For everyone else it means that a large part of the thinking is visible only to the operator. A paper by researchers from OpenAI, Google DeepMind, Anthropic, Meta and other institutions (Korbak et al., 2025) describes the monitorability of chains of thought in natural language as a new but fragile opportunity for AI safety and recommends that developers consider the impact of their design decisions on this monitorability. Latent thinking goes in the opposite direction.

For the openly released models, however, the HybridLink is part of the download, and anyone can generate the pseudo-texts there. The exclusivity argument only holds for closed models.

Code and Models Under the MIT License

Both projects are published on GitHub under the MIT license.

NCM (HazyResearch/ncm) contains the model code and an evaluation script. Tokenizer and generator for every task in the paper, XSum, WMT14, Sudoku at three levels, LM1B in three sizes, TinyStories and OpenWebText, are on Hugging Face together with test data. The repository does not include a training script. Python 3.10 or newer and PyTorch with CUDA are recommended.

HybridAgents (togethercomputer/HybridAgent) contains the code for training, inference and the pseudo-text inspector. The agent models with 8 and 14 billion parameters and the training data for both training stages are on Hugging Face. The web search requires a Tavily API key. For HybridSFT the authors recommend 8 to 16 GPUs, for HybridRL 32 GPUs on four nodes.

Conclusion

NCM is a clean transfer of an image technique to text. A tokenizer breaks text into levels from coarse to fine, a generator fills each level in a single pass, and with small models the result is faster and more economical than the diffusion models it is measured against. The Sudoku result shows where parallel methods are strong overall, in tasks where every position depends on every other. NCM is narrowly ahead there.

There is still a long way to go before use in large language models. Fixed lengths, one model per task and a size far below a billion parameters are not minor issues to be settled in passing. The HybridAgents paper reaches further. When models leave language behind while thinking, faster thinking is bought with less insight.

NCM is a further development, not a finished tool. A sketch of what can come after token-by-token writing.

Sources

• Kumbong, Khare et al.: Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text Generation, arXiv 2610.14162, October 2026
• NCM code and models, GitHub, MIT license
• Zou et al.: Hybrid Reasoning Agents, arXiv 2610.18861, October 2026
• HybridAgent code and models, GitHub, MIT license
• Tian et al.: Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction, arXiv 2404.02905, 2024
• Hao et al.: Training Large Language Models to Reason in a Continuous Latent Space (Coconut), arXiv 2412.06769, 2024
• Korbak et al.: Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety, arXiv 2507.11473, 2025
• OpenAI: Reasoning models, API documentation