What is Agentic AI Security?

Definition

Agentic AI security is the practice of securing autonomous AI systems — agents that plan, reason, and take actions in your environment via tools, APIs, and other agents.

It combines:

• AI security (model safety, prompt injection, output validation).
• Identity security (per-agent identity, least privilege, audit).
• Application security (tool security, sandboxing, network controls).
• Operations security (monitoring, anomaly detection, incident response).

In simple terms:

Agentic AI security ensures that agents do only what they should, only what they're allowed, only when sanctioned — and you can prove it after the fact.

---

Why It Matters

Agentic AI introduces fundamentally new risks:

• Autonomy — agents act without per-step human oversight.
• Non-determinism — same input can produce different actions.
• Tool composition — combinations of benign tools enable harmful outcomes.
• Prompt injection — adversarial text in inputs/data can hijack behavior.
• Speed and scale — agents act in milliseconds across many systems.
• Delegation — trust must propagate cleanly from users to agents to sub-agents.

Without agentic AI security, your most powerful new automation is also your largest insider-risk surface.

---

Threat Model

1. Prompt Injection

Hidden instructions in user input or external content (web pages, emails, documents, retrieved RAG chunks) hijack the agent's behavior. The agent can't reliably distinguish trusted system prompts from untrusted content.

2. Jailbreak

Adversarial prompts bypass the system's safety instructions, causing the agent to perform restricted actions.

3. Excessive Agency

Agent has more tools/data access than the use case requires. A single compromise yields outsized impact.

4. Tool Composition Attack

Each tool benign in isolation; combinations enable attack (read-PII tool + email-send = exfiltration).

5. Multi-Agent / Cascading Compromise

One compromised agent in a multi-agent system propagates malicious instructions to other agents.

6. Data Poisoning

Training data, RAG corpora, or memory tainted to influence future agent behavior.

7. Model Output Exfiltration

Agent echoes secrets / PII into responses (when instructed by injection or by accident).

8. Resource Abuse / DoS

Adversary triggers expensive tool calls or chains; cost / rate / availability impact.

9. Unauthorized Delegation

Agent delegates to other agents or sub-systems beyond its authorized scope.

10. Memory Leak Across Sessions/Tenants

Agent's persistent memory leaks data across users / customers.

---

Core Controls

1. Identity

• Dedicated identity per agent.
• Strong, federated authentication.
• Owner and lifecycle.

2. Authorization

• Tool allow-lists.
• Data scopes (tenants, folders, regions).
• Bounded delegation (OBO with narrow scopes).
• HITL on sensitive actions.

3. Input / Output Hardening

• Trusted vs untrusted text separation in prompts.
• Allow-listed instructions in system prompts.
• Output validation (sensitive data redaction, structured response schemas).
• Tool output sandboxing (don't auto-trust returned content as new instructions).

4. Sandboxing

• Code execution in containers / WASM with no network access by default.
• File system read-only or scratch volumes.
• Time and resource limits.

5. Network Controls

• Egress restricted; only allow-listed destinations.
• No direct internet for tools that don't need it.
• Service mesh / API gateway in front of tools.

6. Monitoring

• Comprehensive audit (prompts, plans, tool calls, results).
• Anomaly detection (volume, scope, patterns).
• Cost and rate alarms.
• Memory / state changes logged.

7. Containment

• Per-agent rate limits.
• Per-agent budgets.
• Kill switch.
• Graceful pause.

8. Governance

• Inventory + risk classification per agent.
• Approval workflow for new agents and significant changes.
• Periodic review (quarterly).
• Decommissioning playbook.
• Tabletop exercises for agentic incidents.

---

Real-World Examples

1. Email Triage Agent Hijacked

An agent reading inbound emails to triage them was hijacked by hidden instructions in an email body. The agent forwarded confidential drafts. Mitigation: separate read-only triage from action agents; HITL for any forwarding; treat email content as untrusted.

2. Retrieval Poisoning

An agent's RAG corpus included an internal wiki page edited by a contractor. Hidden instructions in the page caused the agent to leak data when retrieved. Mitigation: source vetting; provenance tagging; treat retrieved content as untrusted.

3. Multi-Agent Loop Attack

Agent A delegated to Agent B; B back to A. A prompt injection caused infinite recursion until budget cap hit. Mitigation: chain depth + recursion limits; per-conversation budgets.

4. Tool Composition Exfiltration

Agent had read-database, write-CRM, send-email tools. Each scoped narrowly. Together: exfiltration vector. Mitigation: tool composition risk analysis; HITL approval for sensitive combinations.

5. Cross-Tenant Memory Leak

A multi-tenant agent's memory store wasn't tenant-scoped. Tenant A's session referenced Tenant B's data. Mitigation: per-tenant memory partitioning; access tests in CI.

---

Best Practices

1. Treat all input as untrusted — including retrieved content, tool results, files, emails.
2. Per-agent identity, least privilege, bounded delegation.
3. HITL for sensitive actions.
4. Sandbox tool execution.
5. Audit everything (prompts, plans, tool calls, results).
6. Monitor anomalies (volume, scope, time, cost).
7. Containment ready (rate limits, budgets, kill switch).
8. Multi-tenant isolation (memory, prompts, identities, tools).
9. Approve and review agents as a governed asset.
10. Tabletop exercises for agentic incidents.
11. Threat model per agent (input surfaces, tools, data, delegation).
12. Defensive prompt patterns (clear system / user / tool boundaries; instruction shielding).

---

Checklist

• Per-agent identity?
• Tool allow-list and data scopes?
• Bounded delegation (OBO)?
• HITL on sensitive actions?
• Trusted vs untrusted text separation?
• Tool output not auto-trusted as instructions?
• Sandboxed tool execution?
• Comprehensive audit?
• Anomaly + cost alarms?
• Containment ready (rate limits, budgets, kill switch)?
• Multi-tenant isolation?
• Approval workflow for new agents?
• Quarterly review?
• Tabletop exercises?

---

How Forestall Helps

Forestall provides agentic AI security capabilities:

• Discovery and inventory of agents.
• Risk-scoring per agent (over-permissioning, exposure, composition risk).
• Mapping of agent → tools → data → users.
• Audit log integration for anomaly detection.
• Lifecycle and ownership tracking.

---

Frequently Asked Questions

Is agentic AI security the same as LLM safety?

No — LLM safety addresses model behavior; agentic AI security addresses real-world actions agents take.

Do model providers handle this?

Partially — they provide some safety filters; you must add identity, authorization, sandboxing, monitoring, and governance for your environment.

Are MCP servers in scope?

Yes — Model Context Protocol servers are tools agents call; they need authentication, scoping, monitoring like any tool.

How do I start?

Inventory agents, assign owners, classify risk, apply baseline controls (per-agent identity, least privilege, audit, HITL on sensitive actions), then iterate.

Is this a one-team responsibility?

No — Security + Engineering + Product + Compliance + Legal jointly. Designate an Agent Security Lead.

---

Conclusion

Agentic AI is uniquely powerful — and uniquely risky. Agentic AI security combines identity, application, and operations security with model-specific defenses (prompt injection, output validation, sandboxing). Inventory agents, govern them as assets, apply layered controls, and monitor exhaustively. Get this right and agentic AI delivers transformative productivity safely. Get it wrong and you've deployed a fast, smart, unsupervised insider at scale.