A growing share of AI systems are no longer just model parameters queried in isolation. They are stateful systems with prompts, memory, retrieval databases, code harnesses and other mutable pieces that can shape behavior. In a new essay, one researcher argues that this text-based layer should be taken more seriously as a way for models to learn and adapt.
The central claim is that changes to text artifacts can function much like updates to model weights. In both cases, the system takes in new information and alters future behavior. The difference is where the change happens. Instead of adjusting neural parameters directly, developers can modify prompts, memories, retrieved documents or surrounding software to influence how a model responds.
The author says this approach is especially useful when information is volatile, local or not yet worth baking into a model permanently. Examples include search agents that need fresh web context, personal assistants that rely on changing user history, and systems that must keep auditable records of what they learned. In those settings, the text layer acts as a staging ground before knowledge is eventually distilled into weights, if it is distilled at all.
One of the main arguments in favor of text optimization is sample efficiency. The essay says relatively short, high-likelihood text can capture useful patterns with far less data than weight training often requires, particularly when there are only a few examples to learn from. The author frames this as a form of compression, where compact instructions or memory entries can patch a pretrained model’s prior without requiring a full retrain.
The piece also points to a broader trend in AI development. Instead of relying only on training, several teams are using prompts, retrieval and workflow design to expose capabilities already present in models. Over time, some of that behavior may later be distilled back into weights, but the text layer lets developers test ideas first.
The article cites systems associated with Anthropic, OpenAI, Cursor, Letta, Hippocratic AI and Harvey as examples of projects that use text or surrounding infrastructure to help shape model behavior.
The essay’s most notable concept is what it calls update-time compute. The idea is that a system can spend more computation learning from a single experience by revisiting traces, proposing candidate changes, testing them and deciding what to keep.
That is different from standard gradient descent, where updates are typically committed to a single parameter vector. In the text-based approach, the system can explore several hypotheses, compare them against evidence and revise its own plan before making a change. The author says this is similar to how a scientist tests competing ideas before settling on one.
This becomes valuable, the essay argues, when failures are expensive, behavior is hard to specify, or there is a large amount of offline trace data that does not fit neatly into supervised fine-tuning or offline reinforcement learning.
The author also acknowledges strong arguments for keeping learning in the weights. Weights can amortize stable knowledge, reducing the need to restate it in every prompt or context window. The essay agrees that basic, repeatedly useful information often belongs in parameters eventually.
Still, the author argues that text should not be dismissed as a lesser tool. It can be more auditable, easier to revise and better suited to information that changes over time. The essay also notes that text-based methods are vulnerable to sloppy experimentation and benchmark leakage, but says that is a reason for stronger research methods, not for ignoring the approach.
The broader message is that AI research may have overcorrected toward treating weights as the only serious place for learning. In the author’s view, prompts, memory and retrieval systems are not just scaffolding around intelligence. They are part of the learning process itself.