Agentic attacks: the case of independent guardrails in the operating environment of the agent
Agentic attacks are attacks where an agent performs an autonomous exploit sequence involving a multi-stage exploit. There is minimal human involvement, with the human input often limited to an initial prompt to set the goal of the exploit. The agent deconstructs this goal into a series of tasks, representing the individual steps of the exploit, and executes these tasks to achieve the goal.
How are agentic attacks different from human attacks?
Agentic attacks differ from human-driven attacks in fundamental ways. The primary differences include:
- Machine-speed: agentic attacks are autonomous and execute at the speed of the available compute. As an example, an agentic attack propagates from reconnaissance to credential access, lateral movement and to exfiltration by executing the individual exploit steps at compute speeds. Human-driven attacks involve cognitive processing by the human threat actor, limiting the speed to human speeds.
- Dynamic reasoning: agentic attacks leverage the autonomous decisioning and reasoning capabilities of an agent. The agent invokes Large Language Models (LLMs) that have reasoning and inference capabilities to evaluate among a set of choices and reason through the exploit sequence.
- Concurrent attacks and parallelism: agentic attacks involve the agent decomposing multiple steps of the exploit into smaller tasks and launching subagents to execute these tasks in parallel. The concurrency is a unique difference between agentic and human attacks.
Figure: agentic attacks and concurrency evaluation of attack pathways
Attack vectors and MITRE ATLAS
Attackers have evolved AI specific attack techniques. Examples of these techniques are: prompt injection (direct and indirect), AI agent tool credential harvesting, RAG poisoning. The MITRE ATLAS (Adversarial threat landscape for artificial intelligence systems) framework provides a comprehensive taxonomy of AI specific attack techniques.
Figure: ATLAS matrix for AI systems
Defender’s perspective: AI safety and built-in model specific guardrails for AI security
The initial focus on the AI rollout has been anchored around AI safety. This is to ensure that the AI system behaves as intended and stays with the intended boundaries.
AI systems have built-in guardrails that are added by the Frontier AI labs. These guardrails are model-specific guardrails, serving as a form of input filtering or output inspection to identify and filter malicious input or output.
The case of guardrails in the operating environment of the AI model
While the built-in, model specific guardrails and governance principles are essential, the fundamental nature of AI systems indicates that it is not possible to guarantee the model behavior until execution time.
Security teams need to ensure the AI systems function as intended even during the actual execution or model reasoning process. Security principles of defense in depth require the selection of independent security controls that belong to the operating environment of the AI system. A rogue agent, one that is controlled by an AI-assisted attacker or one that exhibits emergent misaligned behaviors, would need security controls that detect malicious activity from the agent.
By deploying deceptions (decoys and honeytokens) in the operating environment, defenders gain the benefit of an independent security layer that changes the ground truth of the environment within which the agent operates and identifies malicious activity from the agent. Deception serves as a critical runtime layer that manipulates the adversary’s world model, raises uncertainty and cost, and complements containment, monitoring, interruptibility, and governance mechanisms. It complements model-specific guardrails through environmental guardrails, serving as an important layer as agents operate autonomously in open environments. This surfaces compromised, prompt-injected, or misaligned agents based on what they interact with, going beyond the classification of their behavior.
Example: cloud service accounts to surface privilege escalation
Consider a case of an enterprise that has deployed AI agents for automation. An attacker performs indirect prompt injection, bypassing the built-in, model-specific guardrails and triggering an offensive sequence that begins with reconnaissance to identify cloud service accounts that represent administrative accounts, and attempting to impersonate the accounts to elevate privileges. The autonomous nature of the exploit constitutes an agentic attack.
Defenders anticipate that the agent that operates in an environment with a large cloud footprint introduces a risk of the agent attempting to target privileged service accounts in the cloud. By deploying honeytokens that represent deceptive service accounts in the cloud, the defender changes the ground truth of the operating environment of the agent.
The agentic attack finds the honeytoken service account, attempts to impersonate this, resulting in an immediate detection alert. The defender has the ability to tarpit the agent by configuring the honeytoken account to lead to a decoy, manipulating the reasoning process of the agent and safeguarding the production cloud workloads.