numbat - AI agent observability, (Fri, Sep 4th)
The Perplexity AI team has released numbat, an open-source tool for observing and managing AI agent activity on desktop and CLI systems. Numbat provides real-time monitoring of agent behavior, detects potential malicious activity (like network scanning), and facilitates incident investigation. It’s designed to address the growing challenge of managing a proliferation of autonomous agents and improve visibility into their actions, particularly in light of concerns about agent sprawl and potential exploitation.
Numbat is a new open-source tool developed by Perplexity AI for observing and managing AI agent activity. It addresses the increasing problem of AI agent sprawl and the difficulty in understanding what these agents are doing on a user’s system. The tool provides real-time monitoring of agent behavior, including detecting potentially malicious activity such as network scanning (using tools like nmap). Numbat works by installing local hooks that monitor agent activity and logs, and it can write findings to a records.ndjson file.
Users can install numbat and then use it to enumerate installed agents, as demonstrated by the example of scanning for agent activity with nmap. The tool includes a rule catalog with numerous detectors, categorized by behavior (secrets, exfiltration, integrity, execution, etc.). These rules can be enforced to prevent or block specific actions. Numbat also offers incident investigation capabilities, including case creation, artifact collection (including SHA256 hashes), and a timeline view of events.
Key features include: real-time agent monitoring, rule-based detection, incident investigation tools, and support for various capture methods (OTLP/HTTP). The tool is designed to help organizations manage and secure their AI agent deployments, particularly as agents become more prevalent and potentially more vulnerable. It offers MDM opportunities for fleet deployments and is a valuable addition to the security landscape as AI agent usage continues to grow.