Anthropic Uncovers Hidden 'J-Space' in Claude Opus LLM, Offering Glimpse into AI Processes
AI firm Anthropic has developed a new technique utilizing a tool called the Jacobian lens, or J-lens, to gain insights into the internal workings of large language models (LLMs). This tool uncovered a hidden area, named the J-space, within its Claude Opus 4.6 model, which contains individual words related to concepts the LLM is actively processing for future responses. This discovery aims to provide a clearer understanding of how LLMs operate before generating output, potentially offering new methods for model understanding and control.
Anthropic, an AI firm, has developed a new technique designed to provide a clearer understanding of the internal processes within large language models (LLMs) as they execute tasks or answer questions.
The company built a tool named the Jacobian lens (J-lens), which was used to uncover a hidden area, designated as the J-space, inside Claude Opus 4.6, a version of Anthropic’s flagship LLM released in February. The J-space holds individual words that correlate with the words and phrases the model is likely to produce in a near-future response. This allows researchers to infer what the model is processing internally before it generates an output, suggesting that an LLM's actual operations can sometimes differ from its stated actions.
Anthropic claims that monitoring the words appearing in the J-space offers a novel approach to comprehend and control its models. The company shared its findings in a paper published on its website this week. Additionally, Anthropic has partnered with Neuronpedia, an open-source platform for exploring LLMs, to create a hands-on demonstration accessible to the public.
Tom McGrath, chief scientist and cofounder at Goodfire, a startup also involved in developing tools for LLM understanding and control, described the work as "very good and interesting." This new technique builds upon Anthropic's ongoing efforts in mechanistic interpretability, a field focused on probing the internal mechanisms of LLMs.
The J-lens is an adaptation of an existing tool called a logit lens, which identifies words an LLM is likely to produce next. Unlike a logit lens, the J-lens highlights words an LLM is likely to express at some point in the near future, not necessarily immediately. This reveals words related to an LLM's current task that may not ultimately be part of its final response.
McGrath noted that while the contents of the J-space are often routine, they can sometimes reveal "quite surprising things that seem to be, like, sort of internal themes or thought processes." Anthropic has provided several examples of these observations.
(Source: MIT Technology Review)
