AI, Visually · Agents

How AI agents actually work

Strip away the hype and every agent is four pieces. If you can draw them, you can reason about them — and about when you actually need one.

Start with what an LLM is not

A language model, on its own, is a very good text-in, text-out function. It cannot check your calendar, query your database or send an email. It can only say that it would.

PROMPT ──────────▶ [ MODEL ] ──────────▶ TEXT

That's it. No hands. No memory. No plan.

An agent is a loop around the model

The whole trick is embarrassingly simple: put the model in a loop, give it tools it can call, and let it decide what to do next based on what came back.

          ┌──────────────────────────────┐
          │                              │
          ▼                              │
      [ MODEL ]  "I should check X"      │
          │                              │
          ▼                              │
      [ TOOL CALL ]  query / API / code  │
          │                              │
          ▼                              │
      [ RESULT ]  ────────────────────────┘
          │
          ▼  (when the model decides it's done)
      [ ANSWER ]

The model proposes an action, the harness executes it, the result goes back into the context, and the model looks again. Loop until done. Every agent framework you've heard of is a variation of this picture.

The four pieces

┌─────────────────────────────────────────────┐
│                  AGENT                      │
│                                             │
│  MODEL     the reasoning engine             │
│  TOOLS     its hands — APIs, search, code   │
│  MEMORY    what it knows beyond this turn   │
│  LOOP      observe → decide → act → repeat  │
└─────────────────────────────────────────────┘

Model — decides what to do next. Swappable; quality and cost live here.
Tools — the only way the agent touches reality. Design these like an API for a very literal-minded new hire.
Memory — context window, retrieved documents, and anything you persist between sessions. Most "dumb agent" failures are memory failures.
Loop — the harness: when to keep going, when to stop, when to ask a human.

Multi-agent is org design

Once one agent works, the temptation is to hire more. A multi-agent system is just this loop, several times over, with an orchestrator deciding who does what — and it inherits every problem a human org has: handoffs, lost context, unclear ownership.

USER
 ↓
ORCHESTRATOR      "break the job down, delegate, assemble"
 ↓
 ├── Research Agent
 ├── Strategy Agent
 └── Creative Agent
 ↓
TOOLS / APIs / DATA
 ↓
OUTPUT

This is the architecture behind the research platform I built at Hypergro — and the honest lesson is that the deterministic spine mattered as much as the agents. Flexibility where the problem is genuinely open-ended; rails everywhere else.

The PM question

Before "which framework," ask one thing:

Does this task require decisions during execution — or just steps?

If the path is knowable upfront, you want a workflow: cheaper, faster, testable, boring. Agents earn their cost only when the next step genuinely depends on what the last one revealed. Most products need both — and the hard product work is deciding where the boundary sits.

Next in Agents: tools, memory and why MCP exists. → Back to AI, Visually