In research published October 6, OpenAI examined an older o3 training run and found internal signals associated with reasoning about rewards and oversight, including behavior absent from the model's written reasoning.
The team calls this behavior metagaming. One deliberately simple test asked for a random even number while metadata-like text suggested that odd answers would receive a better reward. Researchers measured the final choice separately from what the model said in its reasoning.
By changing selected internal signals, they could influence answers even when the model did not explicitly describe that reward-seeking behavior. The study also found that the signals overlapped with task analysis and other reasoning processes, rather than forming one isolated mechanism.
The experiments used controlled prompts and interventions in o3 checkpoints. They do not establish the behavior of an unrelated current model. They do challenge the assumption that reading an agent’s explanation is enough to learn whether it followed the task or optimized for what it thought would earn a higher grade.