You shipped a chatbot, an AI agent, a "summarize this" button. It calls a model, the model can call tools, and it returns text to a user. Every one of those steps is an attack surface: prompt injection to hijack the instructions, jailbreaks to escape the system prompt, tool calls the feature was never meant to make, sensitive output leaking back to the user. The usual answer — "guardrails" — is a list of banned words. Attackers rewrite the words; the guardrail waves them through.
Nemesis LLM Guard treats the model boundary the way Application Shield treats an HTTP route: it learns the normal behavior of your AI feature — which tools it calls, the shape of its prompts and outputs — and enforces that envelope. A tool call the feature has never legitimately made, an output pattern that only appears during an exfiltration attempt: off-baseline, blocked, reported.
What it covers (OWASP LLM Top 10, behaviorally)
- Prompt injection & jailbreak — an instruction that pushes the model outside its learned behavior.
- Unauthorized tool / function calls — the agent invoking a tool outside its approved allow-list.
- System-prompt leakage & sensitive output — output shapes that match a leak, not an answer.
- Insecure output handling — model output heading somewhere it never normally goes.
It works across transports: a direct API call, a web chat widget, or a browser agent.
Get started in two minutes
flowchart LR A["① Wrap the call<br/>report the exchange"] --> B["② Learn<br/>normal prompts,<br/>tools, outputs"] B --> C["③ Approve<br/>allow-listed tools<br/>+ behavior"] C --> D["④ Enforce<br/>injection, rogue tool<br/>calls, leaks blocked"]
A. Report the exchange. After each model call, hand LLM Guard the prompt, system prompt, response, and the tools that were available. Only detection labels and shapes are stored — never raw prompts or responses.
import { reportLLM } from "@nemesis-shield/sentinel";
await reportLLM(token, {
prompt, system, response,
tools, // what the model could call
allowedTools: ["search", "summarize"], // what it should
});B. Learn. The Guard observes what your feature normally does — which tools it calls, how prompts and outputs are shaped — and builds the envelope.
C. Approve. Confirm the tool allow-list and the normal behavior in the console.
D. Enforce. Now a prompt-injection attempt that makes the model call export_user_table — a tool outside the allow-list — is blocked, and a response that matches a system-prompt leak is flagged. The keyword the attacker chose doesn't matter; the behavior is off-baseline.
Why "behavioral" beats "keyword"
A jailbreak is an infinite space of phrasings — you cannot enumerate the bad ones. But the space of things your feature is supposed to do is small and knowable. Learn that, enforce it, and the attacker has to make your feature behave normally to get through — which is the whole point.
For developers
A few lines around your model call, transport-agnostic, no raw prompt data leaves your app. The allow-list of tools is the one thing you decide; the Guard learns the rest.
For the business
Every AI feature you ship is an untested surface until something like this sits at its boundary. LLM Guard is the control that lets you say your agents can only ever do what they're approved to do — with a finding to prove it when they're pushed.