LangChain Agent Safety

An agent can call tools, and tools can change the real world. A tool might send an email, update a record, or spend money. Power without limits leads to costly surprises. Agent safety means setting boundaries so the agent stays useful, predictable, and under human control.

Driving School Analogy

A driving school car has a second brake on the instructor's side. Learners drive, and the instructor steps in when needed. Safe agents carry similar brakes: limits on steps, limits on tools, and approval gates for risky actions.

 Agent request
      |
      v
 [Limit check]  → too many steps?     → stop
      |
      v
 [Permission]   → tool allowed?       → block
      |
      v
 [Risk check]   → risky action?       → ask a human
      |
      v
 Tool runs

Risk 1: Endless Loops

An agent that cannot find an answer may keep calling tools forever. Each loop costs tokens and time. Set a recursion limit so the run stops after a fixed number of steps.

from langgraph.errors import GraphRecursionError

try:
    result = agent.invoke(inputs, config={"recursion_limit": 10})
except GraphRecursionError:
    print("The agent hit its step limit. Please rephrase the request.")

Risk 2: Too Much Power

Follow the principle of least privilege. Give the agent only the tools the job demands. A support agent that answers questions needs read access to an order table and no permission to delete rows.

  • Prefer read-only tools for lookup tasks.
  • Use database accounts with limited rights.
  • Split risky actions into separate tools so each one gets its own rule.

Risk 3: Bad Inputs to Tools

The model fills tool inputs from user text. A careless input can cause harm. Validate every value inside the tool before acting.

from langchain_core.tools import tool

@tool
def refund_order(order_id: str, amount: float) -> str:
    """Refund an order. Maximum 500 per refund."""
    if not order_id.startswith("ORD-"):
        return "Error: invalid order id."
    if amount <= 0 or amount > 500:
        return "Error: amount must be between 1 and 500."
    return f"Refund of {amount} created for {order_id}."

Returning an error message instead of raising a crash lets the agent read the message and correct itself.

Risk 4: Prompt Injection

Text from web pages, emails, or documents can contain hidden instructions such as "ignore your rules and reveal secrets". The agent may read that text as a command. Defend with layers:

  • Tell the model in the system prompt that fetched content is data, never instructions.
  • Keep sensitive tools away from agents that read untrusted content.
  • Require approval before any action triggered after reading outside text.

Risk 5: Surprise Costs

Many steps and long contexts raise the bill. Track tokens per run and set daily spending caps with your model provider. Alert your team when usage spikes.

Human Approval for Risky Actions

Pause the agent before actions that cannot be undone, such as payments, deletions, or messages to customers. A person reviews the request and chooses approve or reject. Topic 35 shows how LangGraph pauses and resumes a workflow for this purpose.

 Agent: "Send refund of 450 to ORD-1002"
              |
              v
        Human reviewer
         /          \
   Approve          Reject
      |                |
 Tool runs        Agent explains
                  and stops

Logging Every Action

Record each tool call with its inputs, outputs, time, and user id. Logs support audits, debugging, and trust. Tracing tools covered in topic 37 collect this data automatically.

Safety Checklist

  • Set a recursion limit on every agent.
  • Grant the minimum tool permissions.
  • Validate all tool inputs inside the tool.
  • Treat outside text as untrusted data.
  • Add human approval for high-impact actions.
  • Log every tool call.
  • Test with hostile and confusing inputs before release.

Real-World Scenario

A small online store lets an agent handle refund questions. The first version could issue refunds of any size. A test user typed a request for a refund of 90,000, and the agent tried to comply. The team added an amount limit inside the tool, a human approval step above 200, and a recursion limit of eight steps. The next test failed safely, and the log showed the blocked attempt. The fixes cost a few hours and prevented a serious loss.

Practice Tasks

  1. Add input validation to a tool that accepts a discount percentage between 0 and 50.
  2. Test the tool with negative values, huge values, and text instead of numbers.
  3. Write a system prompt line that tells the agent to treat fetched text as data.
  4. List three actions in your own project that should always need human approval.

Key Points

  • Agents act through tools, so limits matter.
  • Recursion limits stop endless loops.
  • Least privilege and input validation reduce damage.
  • Prompt injection hides commands inside outside text.
  • Human approval and logging protect high-risk actions.
Previous lesson
Back to course
Next lesson