A new transparency audit of Google DeepMind's DiffusionGemma suggests that the text diffusion model is not meaningfully less transparent than a comparable autoregressive model, even though it works in a different way. Researchers from DeepMind's interpretability and text diffusion teams said the model's outputs remain monitorable and that several of its intermediate states can be interpreted without hurting performance.
The findings matter because transparency is increasingly seen as a key part of AI safety. If a model's reasoning can be inspected, developers may be better able to detect errors, misalignment, or other problematic behavior. The researchers said the work is also meant to set a precedent for evaluating future architectures that carry more of their computation in latent or non-text spaces.
At a high level, DiffusionGemma differs from standard language models that generate text token by token. Instead, it produces a whole canvas of text and refines it over a series of denoising steps. That makes the model look more opaque at first glance, because there are more internal steps between the inputs and the final answer.
The team measured this through what it calls opaque serial depth, a way of estimating how much computation happens between interpretable states. By that measure, DiffusionGemma initially appeared far less transparent than Gemma, the company's autoregressive model. But the researchers said they could apply a logit lens technique to intermediate vectors and replace noninterpretable information with token-level approximations without materially reducing downstream performance. With those intermediates treated as understandable, the gap in opaque serial depth shrank to roughly the same level as Gemma.
The audit separates transparency into two parts. Variable transparency refers to whether a snapshot of the model's internal state can be understood. Algorithmic transparency is harder, and asks whether those snapshots are enough to reconstruct how the model arrived at its output. The researchers argue that DiffusionGemma performs well on the first measure, but still lags on the second.
That distinction is important because diffusion models can revise every token position during each step, which means the causal path through the model may not follow a neat left-to-right order. In their analysis, the researchers say this can produce behaviors that are unusual compared with autoregressive systems, including non-chronological reasoning, token smearing, sequence smearing, and intermediate-context reasoning.
One example involved retroactive self-correction. In a counting task, the model first produced an incorrect answer, then listed the relevant numbers, and later adjusted its earlier output to fix the mistake. Another phenomenon described in the audit was token smearing, where the model seems confident that a token belongs somewhere in the output but spreads that belief across neighboring positions until later steps sharpen it.
The researchers also tested monitorability, a downstream measure of whether a model's behavior can be observed in ways useful for oversight. On that metric, DiffusionGemma performed similarly to Gemma.
The paper includes 24 open problems for follow-up research and argues that transparency audits should become standard for new model architectures, especially as more systems shift parts of their reasoning into latent representations. The authors said future models that rely more heavily on hidden reasoning may require new tools that can translate internal activity back into natural language. They pointed to emerging approaches such as Natural Language Autoencoders and Activation Oracles as promising directions.
For now, the main takeaway is mixed but cautiously reassuring. DiffusionGemma appears more complex internally than a standard language model, but the audit suggests that it can still be monitored and partially interpreted in ways relevant to safety research.