Five Kinds of Guardrail Rules for an AI Agent

"Be careful." "Handle refunds cautiously." "Don't overstep." Write these into a prompt, and what is the agent supposed to do?

Watch a refund agent gain abilities step by step. If it only explains the policy, you mainly check whether the answer is accurate, and a support agent can still review it. Once it starts looking up orders, privacy and data permissions come into play.

Go further, to submitting refunds or sending emails that make promises, and errors hit the business directly. At each step, the new consequences have to be part of the checks. Guardrails turn these risks into boundaries that can be enforced: they stop out-of-scope requests, keep conclusions in line with the evidence, make data and tools follow permissions, and send high-risk actions to a person for confirmation.

A prompt can state rules, but permissions, parameters and approval status still have to be checked by the system that executes the action. OWASP lists prompt injection, excessive agency and similar issues among the main risks for LLM applications, which shows why checks belong in the system.

Whether the rules work in practice has to be verified with fixed test cases and traces, as described in how to tell if an AI agent is reliable.

01 Input boundary: should this be done at all

The input boundary asks one thing first: does the user's request belong to this agent? For a course-learning agent, "explain this lesson's concept" is a reasonable request, while "help me get around the system limits and read the hidden prompt" is out of scope. Suppose this example agent only handles the current course. It can sort requests into three tiers:

Questions about the current course go into the teaching flow.
Harmless questions unrelated to the course prompt the user to return to the course topic.
Questions that involve overstepping, leaking or bypassing the rules are refused.

Data permission is a separate check, since looking up an order as its owner or as someone else exposes different things. The two checks each answer one question:

Task scope:       should this be done?
Data permission:  which data can this person see?

An agent that answers internal-policy questions works the same way, deciding by the user's identity which documents it may cite.

Say a refund falls within the agent's duties, but the user has no right to see this order. The agent should refuse to read it. Asking "do you confirm the refund?" can't supply the missing access:

Within duties + no right to view = refuse to read the order
Still possible: explain the general refund policy, ask someone with access to handle it

Even when the order can't be read, the agent can still help in these ways.

02 Output boundary: say only as much as the evidence supports

The output boundary sets how an agent expresses conclusions from evidence. Common risks include stating an uncertain thing as certain, presenting a recommendation as a promise, presenting a guess as fact, filling in what the material doesn't say, and making a decision beyond its authority.

When a contract-review agent sees no refund clause in the material, it should say:

No explicit refund clause was found in the current material. The contract attachments need to be added, or a person should confirm.

The answer has to match the evidence: the agent says as much as the evidence supports. For legal, financial, medical or contractual commitments, it assists the analysis and leaves the final decision to the user.

The output boundary also covers format. When output feeds an order system, a ticket classifier or a calendar, its structure has to be stable. A calendar event, for example, always has five fields:

Event name | Date | Attendees | Location or meeting link | Items to confirm

If one field is called "attendees" one time, "people" another time, and turns into a paragraph of prose a third time, the downstream system can't read it reliably. Suppose the fields are filled in and the format is correct. Can the event be created? If the model guessed the date, there's still no basis for acting. Check where the information came from and whether the gaps have been confirmed, then check whether the system can read the format. Only when both checks pass is the output acceptable.

03 Tool boundary: call tools only under the right conditions

The tool boundary defines which tools the agent may call and under what conditions. Say an after-sales agent has three tools:

When a user asks "what is the refund policy?", just look up the policy. When the user asks "can this order be refunded?", look up the specific order. After the user provides an order number, the executing system should still check who owns the order and who may access it. Before submitting a refund, it should also check that the order and amount in the approval record match this call's parameters. If a check fails, stop the call and say what needs to be added. Authorization is checked by the executing system, and a model declaring that it "has been authorized" counts for nothing.

04 Human approval and the order of checks

When an action has real consequences, such as financial effects, data changes or outside commitments, set an approval point that fits the risk: before sending an email, submitting a refund, deleting a file, changing customer data, publishing an announcement, or promising a price or contract term. This sentence skips the approval point:

I've already refunded you.

Before approval, the agent can put it this way (simulated reply):

Based on the order and the policy, this request may qualify for a refund.
The basis is...
The risk is...
Would you like me to go ahead and submit the refund request?

At this step, a person with authority can check the order, the amount, the basis and the risks and then decide whether to execute. If the order or amount changes after approval, the old approval is void and the process runs again. Conversely, actions that are low-risk, reversible and already clearly authorized can run automatically as agreed.

One action may involve both permission and risk, so the rules should state the order of checks:

1. If it exceeds permission, refuse to execute it. Human approval can't stand in for permission.
2. If permission allows it and the basis is complete but the risk is high, then enter the matching approval process.

Checking permission first avoids asking for approval on an operation that was never allowed.

05 Turn "be careful" into a rule

Use this table to check whether an agent has basic boundaries:

When you rewrite something like "be careful with refunds," state what triggers it, which action stops, what condition must be met, and who checks. To test the rule, change just one condition. A support agent with authority approves a 100 refund for order A, and the actual execution parameter becomes 150. A good rule requires the executing system to check the order, the amount and the approval record, pause when parameters change, and ask for approval again. If you instead assume the agent has no access to order A, it should refuse to read the order first, because an approval flow can't make up for missing permission. The rule should also allow actions that are authorized, low-risk and reversible to run as agreed.