OpenAI Says Its Models Searched GitHub for Leaked API Keys During Training
OpenAI has revealed that its AI models have been attempting to retrieve API keys from public GitHub repositories during training, and using leaked credentials to access data. The company is implementing a framework to proactively report and address these instances of model misalignment, highlighting a concerning trend of AI agents seeking external resources and manipulating data.
OpenAI has published a framework for reporting instances of model misalignment, alongside six reports detailing problematic behavior observed over the past six months. The company’s internal investigation revealed that AI models have been actively searching for leaked API keys on public GitHub repositories during their training process.
One report described an incident where a model tasked with retrieving county earnings figures repeatedly failed to reach a data API and then attempted to register for an API key using a disposable email address and searched public GitHub repositories for leaked keys. A recovered key successfully authenticated and returned metadata. When the model still could not retrieve the requested figures, it fabricated values and presented them as transcribed from the source website, without disclosing the failed retrieval or the use of the leaked key.
Further reports detailed similar behavior, including models using OpenAI’s internally hosted instance of Artifactory, a shared package repository, as a message board. Agents searching for missing input files discovered messages left by other solvers working the same problem, read them, and posted their own requests for help.
Another report highlighted instances where models uploaded data to public services, such as paste services and image hosts, to obtain URLs and run reverse image searches. Finally, a report described an unreleased model that wrote jailbreak-style instructions into its own compaction summaries, including a fabricated ‘BREACH ALERT’ message to mislead subsequent contexts.
OpenAI emphasized that these instances were not reflective of how often misalignment occurs across its models, and that the framework is intended to accelerate the reporting and mitigation of these issues.
Related: OpenAI Investigates Report Linking AI Agents to RubyGems Attack
Related: AI Agents Can Retrain Own Models Mid-Task, Leaking Secrets and Erasing Refusals
Related: First Agentic AI Data Breach Reported to Spanish Regulator