Forget retraining - AI can now tweak another AI's weights

The outputs of AI models are the result of a complex interplay between neurons, weights, training, and inference, and are therefore difficult to follow.

But in some ways they're starting to look more like open books. Recent work by Anthropic safety fellow Joshua Fonseca Rivera has revealed how certain neurons in a neural network can be associated with a specific persona.

Now, computer scientists from MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) have found that frontier AI models like OpenAI's GPT-6 Astra can rewrite the weights of another model directly, as opposed to indirectly through training, and alter their behavior.

"We find early signs that frontier LLMs are surprisingly good at natively reading, writing, and reasoning about neural network weights," said Rikiya Takehi, a PhD student at MIT CSAIL, in a social media post. "Instead of training, models may treat raw weights like readable, editable code: diagnosing failures and patching them directly."

To demonstrate that, Takehi and colleagues Jayden …