"Agent" is the most overused word in AI right now. It's on every landing page and in every pitch deck, attached to everything from a basic chatbot to a nightly script. That vagueness is a problem, because the difference between a real agent and a plain automation isn't marketing — it changes what you build, what it costs, and how often it breaks.
Here's a practical definition, and an honest look at when you actually need one.
The simplest useful definition
An AI agent is a system where a language model decides its own next step. You give it a goal and a set of tools, and it runs in a loop: look at the situation, choose an action, take it, look at the result, and repeat until the job is done.
The key word is decides. In a normal program, you — the developer — write the control flow. If this, do that; then this; then stop. In an agent, the model chooses the control flow at runtime. You don't know in advance how many steps it will take or which tools it will use, because that depends on what it finds along the way.
That single property — the model controls the loop — is what separates an agent from everything else people call an agent.
The spectrum, from least to most agentic
Most "AI features" fall on a spectrum. It helps to see where the line actually is.
1. A single model call. Prompt in, text out. Summarize this email. Classify this ticket. Draft this reply. No tools, no loop, no decisions about what to do next. This is not an agent, and the large majority of useful AI features live right here.
2. A chain (a fixed workflow). Several model calls wired together in an order you decide. Extract the fields, then validate them, then look them up in a database, then write a summary. The model does the smart parts, but the steps and their order are hard-coded. Also not an agent — and that's a feature, not a shortcoming.
3. A router. One model call decides which of a few predefined paths to take — "is this a refund request, a bug report, or a sales question?" — and each path is then a fixed workflow. Slightly more dynamic, still mostly under your control.
4. An agent. The model gets a goal and tools and runs its own loop. It might call a search tool, read the results, decide it needs to look somewhere else, call another tool, and stop only when it judges the task complete. Number of steps: unknown up front. Path: decided at runtime.
The important insight is that being "agentic" is a dial, not a switch. More agency means more flexibility — and more cost, latency, and unpredictability. You want the least agency that solves your problem.
The anatomy of an agent
When you do build one, four parts show up every time:
- A model that's good at reasoning and reliable at calling tools.
- Tools — functions the model can call to affect the world: search, read a file, query a database, send an email, run code. Tools are where an agent gets both its power and its risk.
- A loop that feeds each tool's result back to the model so it can decide the next move.
- A stopping condition — the task is done, a step budget is hit, or a human is asked to step in.
Notice how much of that is ordinary software. The model is one component. Most of the engineering — and most of the bugs — live in the tools, the loop, and the guardrails around them.
When you don't need an agent
This is the part the hype skips. Agents are the right tool far less often than the marketing suggests. Reach for something simpler when:
The steps are known. If you can draw the flowchart, build the flowchart. A fixed workflow that does the same five steps every time is cheaper, faster, and dramatically easier to debug than a model deciding those steps on the fly. Determinism is a feature. When something breaks, you want to know exactly which step failed — not re-read a transcript of the model's reasoning to reconstruct what it decided to do this particular time.
The task is one shot. Summarize, classify, extract, translate, rewrite — these are single model calls. Wrapping a loop around a task that never needed a second step just adds latency and cost for nothing.
Reliability matters more than flexibility. Every extra step an agent takes is another chance to go wrong, and errors compound: a bad decision on step two poisons every step after it. Imagine a ten-step agent that gets each individual step right 95% of the time — it still finishes the whole task correctly only about 60% of the time, because the odds multiply. Fixed workflows don't have this problem, because the steps can't derail each other.
You can't afford surprises. Agents are non-deterministic by design. The same input can produce different paths on different runs. That's fine for research and exploration; it's a nightmare for a process that has to be auditable, repeatable, or compliant.
A good rule of thumb: start with a single call. If that's not enough, build a fixed workflow. Only reach for an agent when the task genuinely can't be expressed as a fixed sequence of steps.
When you actually do need one
Agents earn their complexity when the path can't be known in advance. Specifically:
The number of steps varies and can't be predicted. "Investigate why this deployment failed" might take two steps or twenty, depending on what the logs say. You can't hard-code that flowchart, because the flowchart depends on the answers.
The task requires exploration. Debugging, research across many sources, or working through an unfamiliar codebase all involve looking at something, forming a hypothesis, and deciding where to look next based on what you found. That look–decide–look loop is exactly what an agent is for.
The action space is too large to enumerate. If there are hundreds of possible tools or paths and only a handful are relevant to any given request, letting the model pick beats writing a router with hundreds of hand-coded branches.
Notice these all share a shape: you genuinely cannot write the control flow ahead of time, because it depends on information you only get while the task is running. That's the real test. If you can write the flowchart, you don't need an agent.
The costs you're signing up for
If you do decide an agent is warranted, go in clear-eyed about the trade-offs:
- Cost. An agent makes many model calls per task, and each call re-sends a growing history. Cost scales with the number of steps — and the number of steps isn't fully in your control.
- Latency. Steps are sequential; each one waits on the previous one's result. A task that's a single call in a workflow can become a slow, multi-second chain in an agent.
- Debuggability. When a workflow fails, you get a clear point of failure. When an agent fails, you get a transcript to read — and the failure might not reproduce on the next run.
- Security. An agent is only as safe as the tools you give it. A tool that can delete records, spend money, or send email is a tool the model can invoke for the wrong reasons — especially if any of its input is attacker-controlled. Scope tool permissions tightly, and put a human in front of anything irreversible.
None of these are reasons never to build an agent. They're reasons to build one deliberately — with a budget on steps, logging on every tool call, and tight permissions — rather than because "agent" happened to be on the roadmap.
The bottom line
An AI agent is a language model running its own loop over a set of tools, deciding its next step as it goes. That autonomy is genuinely powerful for open-ended, exploratory tasks where you can't script the path in advance. For everything else — which is most things — a single model call or a fixed workflow is cheaper, faster, safer, and easier to trust.
The best engineering question isn't "how do we build an agent?" It's "what's the simplest thing that solves this?" Often the answer isn't an agent at all — and knowing the difference is what keeps a project from collapsing under its own cleverness.
Weighing whether your problem needs an agent or something simpler? Get in touch — we help teams design and build AI systems that fit the job.