Frontier AI: The Genie's Out of the Bottle, But Where's the Rulebook?

Several states – Illinois, New York, and California – are enacting laws to regulate the deployment of increasingly powerful frontier AI models, requiring developers to establish comprehensive AI frameworks addressing risk assessment, cybersecurity, and transparency. These laws stem from concerns about the potential for AI models, like those capable of autonomously exploiting vulnerabilities (e.g., Mythos), to be used in critical infrastructure and potentially launch AI-driven ransomware attacks.

Illinois governor JB Pritzker recently signed Senate Bill 315 (SB315), the Artificial Intelligence Safety Measures Act, while New York and California have also enacted similar disclosure laws. These laws aim to address the rapid advancement of AI capabilities, which have expanded beyond simple chatbots to models capable of independently identifying and exploiting zero-day vulnerabilities.

AI disclosure laws continue to emerge because there was no regulation for frontier models, explains Sachin Jade, chief product officer at Cyware. Now that they’ve been circulating, risks have become apparent and people realize they were never "too big to fail," he says.

Critical infrastructure organizations, including the U.S. government, are increasingly using these models in their environments. Models continue to access higher value information developers and companies use for training, and humans are less in the picture. One report recently cited the first AI-executed ransomware attack.

Illinois, California, and New York have similar core requirements, but details vary on features such as third-party audits and disclosure timelines, which will create patchwork compliance. While Trump's latest executive order addressed frontier AI security and included a voluntary framework for the private sector, the White House has not released any formal regulations.

Illinois and New York allow frontier AI developers to report critical safety incidents to the agency and attorney generals within 72 hours of discovery, while California provides 15 days. Windows shrink to 24 hours if the "incident poses an imminent risk of death or serious physical injury."

Despite the progress, significant gaps remain regarding downstream users and the broader implications of these rapidly evolving technologies. For example, some models are open source and designed for users to develop bots and agents, so what happens if someone modified their own model based on a frontier model? "What if I've engrained this model into my ecosystem or my workflow? What do I do?" he poses. "Laws don't address that yet."

To address these challenges, experts advise enterprises to return to the core essence of security – maintaining strong visibility to reduce shadow AI risks, conducting regular audits, application mapping, and implementing identity and access management controls to understand access levels in order to comply. One challenge is that people think tools will solve security, but security is very much a mindset and a culture.