Agentic AI Isn't Risky; the Way Orgs Deploy It Is
This article highlights a critical cybersecurity risk associated with the rapid deployment of agentic AI, focusing on vulnerabilities stemming from poor software development practices rather than inherent flaws in the AI itself. The core issue lies in the interaction between deterministic software tools and large language models (LLMs), creating opportunities for exploitation through weaknesses like insufficient input sanitization and exposed credentials. Recent examples, including vulnerabilities in Salesforce and ServiceNow, demonstrate how attackers can leverage these misconfigurations to gain unauthorized access and control.
The increasing adoption of agentic AI presents a significant cybersecurity challenge, largely due to organizations’ tendency to treat these systems as ‘black boxes.’ While the AI models themselves aren't inherently risky, the way they’re integrated with traditional software and the resulting coding practices create vulnerabilities. The article emphasizes that the primary risk isn’t the AI’s intelligence, but rather the overlap between deterministic software tools and LLMs, leading to exploitable weaknesses. Recent incidents, such as the Salesforce vulnerability triggered by a malicious prompt and the ServiceNow exploit leveraging a default credential, underscore this point. These cases demonstrate how attackers can manipulate the interaction between these components to gain unauthorized access and control within the affected systems.
The article’s key argument is that organizations are often failing to adequately understand the underlying architecture of agentic AI systems, leading to the creation of insecure software. Specifically, the juxtaposition of deterministic systems (traditional software) with non-deterministic LLMs creates a dangerous intersection. This is compounded by practices like relying on LLMs to handle tasks without proper input sanitization or implementing robust access controls. The author, Eliad Kimhy, stresses the importance of understanding how these systems work at a fundamental level to avoid writing insecure software and relying on LLMs for critical functions without proper oversight.
Furthermore, the article points to the misconfiguration of systems, such as Salesforce’s reliance on an expired domain, as contributing factors to the vulnerabilities. These instances highlight the need for a more cautious and deliberate approach to deploying agentic AI, prioritizing secure coding practices and thorough system configuration over simply adopting the latest technology.