Inside the Agent Loop: How an AI Agent Actually Decides What to Do

The first agent I shipped made 41 tool calls trying to answer a question that needed two. It never crashed. It never errored. It just kept politely deciding that one more search would clear things up, and by the time I noticed, it had spent nineteen minutes and a genuinely upsetting amount of money being confidently unhelpful.

Nothing in the tutorial had prepared me for that, because the tutorial's agent had three tools and one test question. What I had built was a while loop with a language model inside it and no adult supervision.

This post is the thing I wish I had read that week: what an agent loop actually is, the five decisions inside it, and the guardrails that turn it from a plausible demo into something you can point at production traffic.

The mechanism

An agent is a loop, and that is genuinely the whole idea

Strip away every framework and an agent is about twelve lines. Everything else in this post is a guardrail bolted onto one of them.

def run(goal: str, tools: dict, max_steps: int = 8) -> str:
    messages = [system_prompt(), {"role": "user", "content": goal}]

    for step in range(max_steps):
        reply = model.chat(messages, tools=schemas(tools))  # THINK
        messages.append(reply)

        if not reply.tool_calls:                    # STOP?
            return reply.content                    # ...answered

        for call in reply.tool_calls:               # ACT
            result = tools[call.name](**call.arguments)
            messages.append({                       # OBSERVE
                "role": "tool",
                "tool_call_id": call.id,
                "content": serialise(result),
            })

    return "Could not finish within the step budget."   # GIVE UP

That is it. The model does not "have agency" in any mysterious sense — it emits a structured request, your code runs a function, and you hand the result back so it can emit the next one. The loop is yours. The intelligence is rented; the control flow is entirely your responsibility.

1. Think — one model call, nothing more

   The model sees the goal, the tool schemas, and everything observed so far. It returns either a final answer or a request to call one or more tools. This step is stateless; all the memory is in what you passed it.

2. Stop? — the most under-designed line in most agents

   "No tool calls" is the happy termination. There are three unhappy ones, and if you have not written them you will discover them in production.

3. Act — your code, your blast radius

   The model does not execute anything. It names a function and supplies arguments. Whether that function can delete a row is a decision you already made when you registered it.

4. Observe — the result goes back as context, and context costs money

   Every observation is appended and re-sent on the next turn. A tool that returns 8 KB of JSON is not a free call; it is a tax on every remaining step in the run.

5. Give up — visibly, with what it has

   Hitting the step budget is a normal outcome, not a crash. Return the partial work and say so. An agent that silently returns nothing is indistinguishable from a hung request.

Where behaviour actually comes from

The tool schema is the prompt

This took me embarrassingly long to internalise. I spent three days rewriting the system prompt to stop the agent over-searching. The fix was one sentence — in a tool description.

The model chooses tools almost entirely from their names, descriptions and parameter docs. That schema is not plumbing you generate and forget; it is the highest-leverage prompt surface in the whole system.

@tool
def search_docs(query: str, section: str | None = None) -> list[Passage]:
    """Search the Laravel and Filament documentation.

    Returns up to 6 passages, each with its source URL and heading path.
    Covers official framework documentation ONLY — not Stack Overflow,
    not changelogs, not application source code.

    If the first call returns nothing relevant, do not rephrase and retry:
    the answer is very likely not in this corpus. Say so instead.

    Args:
        query:   A natural-language question. Full sentences beat keywords.
        section: Optional filter, one of "eloquent" | "filament" | "livewire".
    """

Three sentences of that docstring exist purely to stop a loop I watched happen. That is what tool design is: encoding the lessons of your traces into the one place the model reliably reads.

The resource nobody budgets for

State: the context window is a budget, not a memory

Every turn re-sends everything. The system prompt, the tool schemas, the full message history, and every observation collected so far. This is the part that surprises people coming from ordinary backend work, where state lives somewhere and you fetch what you need.

Here is the same run measured turn by turn. The bars are tokens, as a percentage of an 8k working budget:

Growth is superlinear in practice, because the observations that accumulate are usually the largest messages in the run. Turn 6 is where the original agent started behaving strangely — not failing, just getting vaguer, because the early instructions were now a small fraction of a very long conversation.

The fix is compaction: once history crosses a threshold, summarise the middle and keep the ends.

def compact(messages: list[Message], limit: int = 6000) -> list[Message]:
    if count_tokens(messages) < limit:
        return messages

    head = messages[:2]     # system prompt + the original goal. Never drop these.
    tail = messages[-6:]    # the last three turns, verbatim — the model is mid-thought.
    middle = messages[2:-6]

    summary = model.chat([
        SUMMARISE_PROMPT,
        {"role": "user", "content": render(middle)},
    ]).content

    bridge = {"role": "assistant", "content": f"[earlier steps] {summary}"}
    return head + [bridge] + tail

Knowing when to quit

Termination: four exits, and only one of them is good

The tutorial loop has one exit. A real one needs four, and three of them should log loudly.

That last one caught my 41-call disaster in six lines:

seen: set[str] = set()

for call in reply.tool_calls:
    signature = f"{call.name}:{json.dumps(call.arguments, sort_keys=True)}"
    if signature in seen:
        # Do not just break — tell the model WHY, or the next turn repeats it.
        messages.append(nudge("You already ran that exact call. "
                              "Use what you have, or say you cannot answer."))
        break
    seen.add(signature)

Note that it does not silently break. A guard that stops the agent without telling the agent produces a confused final turn; a guard that explains itself produces a clean "I don't have enough to answer this."

The part that makes it shippable

Guardrails: five gates, any of which can stop a run

The consequence gate deserves its own rule, because it is the one that stops an agent being a liability. I classify every tool at registration time:

What goes wrong

The five failure modes, and how each one announces itself

Every one of these is quiet. Not one throws an exception. That is the defining property of agent bugs and the reason the observability layer stopped being optional for me: an agent that fails loudly is a normal engineering problem, and an agent that fails politely is not visible at all without traces.

run 8f3a2c · goal: "does Filament support nested resources?"

  step 1  think    421 tok   →  search_docs(query="nested resources filament")
          observe    0 hits
  step 2  think    602 tok   →  search_docs(query="filament nested resource")
          observe    0 hits
  step 3  think    780 tok   →  search_docs(query="nested resources in filament v3")
          observe    0 hits
          ^^ REPETITION GUARD would fire here — args differ, intent identical
  step 4  think    966 tok   →  search_docs(query="filament relation manager nested")
          observe    6 hits   ← finally, but 4 steps and 2,769 tokens in

  outcome  answered · 5 steps · 3,441 tok · $0.019 · 6.2s

The repetition guard on exact arguments would not have caught steps 1–3 — the strings differ. What fixed it was the tool description telling the model that an empty first result means the corpus does not cover it. Guards catch the mechanical repeats; good descriptions prevent the semantic ones.

Before you point it at traffic

The checklist

• A hard step ceiling — a number, in code, not a hope.
• A per-run cost cap that stops the loop, not just alerts on it.
• A repetition guard that tells the model it looped.
• Tool descriptions that state what the tool cannot do, not just what it can.
• Argument validation on every tool, with the error returned as an observation.
• Compaction that pins the system prompt and the original goal, never a naive sliding window.
• Irreversible actions behind a human, with no confidence-score escape hatch.
• A trace per run recording every step, token count and dollar figure.
• An answer path that can refuse when the tools came back empty.
• A termination reason on every run, so "answered" versus "gave up" is a metric and not a guess.

Ten items. Nine of them are twenty lines or fewer. Together they are most of the distance between the loop at the top of this post and something you would let near a customer.

Where to start on Monday

Write the twelve-line loop yourself before you install anything. Not because frameworks are bad — LangGraph and CrewAI earn their place the moment you have several coordinating agents or a long-horizon run — but because every one of them is an opinion about the loop, and opinions are much easier to evaluate once you have held the thing they are opinions about.

Then add exactly one guard, the repetition detector, and watch your traces for a day. It will show you your second guard, and your tools will tell you the rest.

• tool schemas
• step budget
• cost cap
• repetition guard
• compaction
• consequence classes
• traces
• termination reasons

If you are building one of these, the production RAG guide covers the retrieval half properly, and the concepts map is worth reading first if the words "agent" and "agentic" still feel interchangeable. And if your agent has found a failure mode that is not in that table of five, I would genuinely like to hear about it — the comments are open.