fire

Black Box

Neuron

Former OpenAI researcher Nick Cammarata argues that about 90% of neurons in a large AI model have one main job, undercutting a theory he helped build.
<blockquote>
 <h2 id="key-points">Key Points:</h2>
 <ul>
 <li>Cammarata says famous examples of tangled neurons were largely misread.</li>
 <li>Some prominent researchers say the findings leave the broader picture unchanged.</li>
 <li>A technical companion paper backing the claims has not yet been released.</li>
 </ul>
</blockquote>
<h2 id="cammarata-s-neuron-audit">Cammarata's Neuron Audit</h2>
Cammarata, now a researcher at the interpretability lab Goodfire, <a href="https://already-clean.goodfire.com/" rel="noopener noreferrer nofollow">laid</a> out the case on Oct. 15 in an essay titled "Neural Networks Are Already Clean." He <a href="https://cryptobriefing.com/nick-cammarata-neural-networks-interpretable/" rel="noopener noreferrer nofollow">argues</a> that most neurons in models with up to trillions of parameters are now legible, contrary to what earlier studies suggested. The claim has <a href="https://the-decoder.com/is-ai-interpretability-close-to-a-breakthrough-a-new-essay-has-researchers-divided/" rel="noopener noreferrer nofollow">divided</a> researchers.
In preliminary audits of the Qwen3-32B model, Goodfire estimates that roughly 90% of neuron poles have one main job, about 7% have two related jobs and only 2% to 4% remain unexplained. AI agents did most of the analysis.
Cammarata also revisits the field's most famous "polysemantic" neuron, found in the image model InceptionV1, which seemed to fire on both cat faces and car fronts. He now reads it as a V-shape detector. Pointed shapes such as fox snouts and flames switch it on, while round shapes like the letter O push its activation negative.
Also Read:<a href="https://yellow.com/news/tesla-flying-roadster-legal-no-clear-answer" rel="noopener noreferrer nofollow">Is Tesla's Flying Roadster Even Legal? No Clear Answer Before 2028</a>
<h2 id="nanda-questions-cammarata">Nanda Questions Cammarata</h2>
Roon, a pseudonymous OpenAI employee who says he is close friends with Cammarata, wrote that the work could go down as a landmark in scientific history. Even he cautioned that others still need to confirm the results.
Neel Nanda, who leads the interpretability team at Google DeepMind, sounded far more measured. He wrote that the essay is interesting but leaves the broader picture largely unchanged, and he predicts that studying neurons alone will have serious blind spots. Nanda still believes superposition is real, arguing that language models hold far more facts than they have neurons, and he doubts that results from small image models transfer to larger systems.
If Cammarata is right, researchers could look for deception or dangerous behavior directly in a model's circuits instead of relying on chains of thought, which are becoming harder to trust. Still, the technical companion paper remains unreleased, and Goodfire, as an interpretability company, has a clear stake in the thesis.
The essay marks a sharp reversal for Cammarata. In 2020 he co-wrote "Zoom In" with Chris Olah, a paper that <a href="https://distill.pub/2020/circuits/zoom-in/" rel="noopener noreferrer nofollow">introduced</a> the first circuit researchers read as superposition, the idea that networks short on neurons pack unrelated concepts together. Three years later, an automated interpretability project he started at OpenAI found only about 1.7% of neurons understandable, even by its most generous measure.
Read Next:<a href="https://yellow.com/news/chatgpt-outscores-claude-haiku-tailored-advice" rel="noopener noreferrer nofollow">ChatGPT Outscores Claude Haiku 5.5 In 3-2 Test, Loses On Tailored Advice</a>

Is AI’s Black Box Cracking? Ex-OpenAI Researcher Says 90% Of Neurons Do One Job

Table of Contents

Yellow News

Other articles published on Oct 18, 2026

Join our free newsletter for daily crypto updates!