Mythos Vulnerability Firehose Hits a Human Bottleneck
A recent analysis by VulnCheck reveals a significant bottleneck in the vulnerability research process, despite Anthropic’s Project Glasswing generating a massive number of AI-identified flaws. Only a small fraction (around 10%) of these findings have been disclosed or patched, suggesting that human validation and coordination are now the primary obstacles in translating AI-generated vulnerability reports into actionable fixes. The study highlights a discrepancy between AI’s aggressive severity assessments and maintainer’s evaluations, raising concerns about the accuracy and reliability of AI-driven vulnerability discovery.
A new analysis by VulnCheck has highlighted a significant gap between the volume of vulnerability findings generated by Anthropic’s Project Glasswing and the actual number of flaws that are successfully addressed. The research indicates that human validation and coordination are now the primary bottlenecks in the vulnerability research process, despite the AI model’s ability to rapidly identify potential weaknesses.
Anthropic’s Project Glasswing has produced a total of 26,153 vulnerability findings across numerous software projects since its launch in April 2026. However, only 2,736 of these findings, or approximately 10%, have made it into the vulnerability disclosure ledger, meaning they have either been disclosed to the appropriate software maintainer or are currently in the process of being disclosed. Less than 0.8% (202) of the findings have been patched, and 245 were withdrawn. Another 191 vulnerabilities were in the pre-disclosure stage and had not been reported to their maintainers yet.
The remaining, nearly 90% of Claude Mythos-generated findings, have not made it to the ledger yet, suggesting that human validation and coordination have become a critical bottleneck. VulnCheck’s analysis comes as Project Glasswing and similar efforts to apply AI to vulnerability discovery across the industry have begun producing a massive volume of newly identified flaws, creating new challenges for security and vulnerability remediation teams.
Contrast Security’s research further demonstrated significant variability in the results produced by AI-powered security scanners. Different runs of the same tool against the same codebase produced substantially different findings, and multiple scanners agreed on only a small percentage of vulnerabilities. Jeff Williams, founder of Contrast Security, noted that three different AI scanners, run three times against the same 50,000-line codebase, generated different results each time. A Sonnet-based simple scan reproduced only 17% of its own findings across the three runs, while Opus reproduced 25% but had a nearly 30% swing in finding count between best and worst runs.
Williams emphasizes that while AI can make finding potential flaws relatively inexpensive, determining which findings are real, actionable, and worth fixing requires substantially more time and resources. He points to Microsoft’s record-setting September Patch Tuesday, which addressed 974 vulnerabilities, as an example of the scale of the challenge organizations face as AI accelerates vulnerability discovery. “The world is still doing security at human speed,” he says. “And that is Anthropic with unlimited compute, and Apple, Google, and Microsoft as partners.”
