AI ENGINEERING: DEEP DIVE · MODULE 06 — AGENTS & ORCHESTRATION INTERNALS · 16 A hands-on series · math + code

Module 6.1

There is no loop in the model

AI Concepts Intermediate 4.2 showed the think-act-observe ring as a diagram. Here's the actual text — every line tagged by who wrote it: the developer's instructions, the model's own output, or code splicing in a result.

GROWING TRANSCRIPT — ONE TEXT BUFFER, RESUBMITTED EACH TIME
Step 0 of 7
Developer-written (fixed) Model-generated Spliced in by code

The math

The "loop" is string concatenation

Each call
completion = model(buffer)
the model sees the ENTIRE buffer, fresh, every single call
no hidden state carries over between calls
Between calls
buffer ← buffer + completion + observation
code parses "Action:", runs it, appends "Observation:"
the buffer only ever grows
Stop condition
stop when completion contains "Final Answer:"
a string match, not a model API flag
the model decides by what it writes, not a signal

Module 1.3 showed one attention step; Module 2 showed one forward pass. A ReAct "agent" is just that forward pass, called repeatedly on a buffer that code keeps growing.

The code

The actual harness

react_harness.py
buffer = SYSTEM_INSTRUCTIONS + f("\n\nQuestion: {question}\n\n")

while True:
    completion = model.generate(buffer, stop=["Observation:"])
    buffer += completion

    if "Final Answer:" in completion:
        break                                # the model decided it's done

    action = parse_action_line(completion)       # regex on "Action: tool(args)"
    result = run_tool(action)                    # your code, not the model
    buffer += f("Observation: {result}\n")   # spliced back in, verbatim

Notice stop=["Observation:"] — the model is explicitly cut off right where it would try to hallucinate its own observation instead of waiting for the real one.

The Nimbus thread

Why the stop sequence actually matters

Without stop=["Observation:"], nothing prevents the model from writing a plausible-looking fake observation itself — "Observation: quota was fine, must be a network issue" — and confidently reasoning from a number it made up. The stop sequence forces a hard handoff: the model proposes an action and nothing past it; only your code is allowed to write what actually happened.

So what

An agent's "memory" of what it just did is a bigger text file — nothing more exotic is happening between one call and the next.