Execution State and Scratchpad¶

What is execution state?¶

During an agentic run, the system state changes continuously:

current goal
current step
completed steps
pending work
observations
artifacts
budget usage
approvals
errors

If this is left only inside the model's conversation context, the runtime becomes fragile.

The important mental model is:

Model context is a current view. Canonical execution state belongs to the application runtime.

Canonical Run State
        ↓ projection
Model Context
        ↓
LLM decision
        ↓ validated transition
Canonical Run State

State vs context vs scratchpad vs memory¶

These concepts should remain separate.

Execution state¶

The authoritative operational state of the current run.

For example:

{
  "run_id": "run-1842",
  "status": "RUNNING",
  "goal": "Fix failing payment test",
  "current_step": "run_regression_tests",
  "completed_steps": ["inspect_failure", "patch_code"],
  "tool_calls_used": 7,
  "cost_used_usd": 0.84
}

Context¶

The information selected for one model call.

relevant goal
+ current state summary
+ latest observations
+ allowed tools
+ relevant code/docs

The model does not need every field of the complete state on every call.

Scratchpad¶

Short-lived working artifacts or explicit intermediate notes.

Examples:

suspected root causes
candidate files
current hypothesis
temporary comparison table

A scratchpad can be runtime-managed, but it is not necessarily authoritative domain state.

Memory¶

Information that survives beyond the current run.

Examples:

user preference
previous incident lesson
repository convention
long-term project fact

That belongs to a separate memory architecture concern.

execution state = this run
memory          = beyond this run

What should be canonical?¶

Usually, anything on which the runtime builds hard control:

run status,
goal and success criteria,
iteration counter,
budget usage,
completed/pending steps,
tool-call result references,
approval state,
side-effect idempotency keys,
checkpoint version.

Do not reconstruct these from model prose such as:

"I think I already ran the tests earlier."

Prefer explicit state:

{
  "test_execution": {
    "id": "test-781",
    "suite": "payment-regression",
    "status": "PASSED",
    "completed_at": "..."
  }
}

State-machine mental model¶

A run can be modeled as a state machine.

For example:

CREATED
  ↓
RUNNING
  ├──→ WAITING_FOR_HUMAN
  ├──→ WAITING_FOR_EXTERNAL_EVENT
  ├──→ FAILED
  ├──→ CANCELLED
  └──→ COMPLETED

Transitions may be triggered by:

a model decision,
a tool result,
a deterministic rule,
user input,
a timeout,
an external event.

But the runtime performs the transition.

LLM proposes: STOP_SUCCESS
        ↓
runtime checks success contract
        ↓
COMPLETED or continue

A possible RunState¶

Conceptually:

{
  "run_id": "run-1842",
  "status": "RUNNING",
  "goal": {
    "description": "Fix flaky retry test",
    "success_conditions": [
      "targeted test passes 20 consecutive runs",
      "related regression suite passes"
    ]
  },
  "plan": {
    "version": 3,
    "steps": []
  },
  "observations": [],
  "artifacts": [],
  "approvals": [],
  "budget": {
    "iterations": 6,
    "tool_calls": 11,
    "cost_usd": 1.18
  },
  "version": 27
}

The exact schema is system-specific; explicit state ownership is the important principle.

Project state into model context¶

Complete state can be large and noisy, so use a separate projection step:

Canonical state
     ↓
Context Builder
     ↓
Relevant state projection
     ↓
LLM

The model may only need:

Goal: fix flaky retry test
Current step: verify patch
Latest observation: target test passed 10/10
Remaining success criterion: related regression suite not yet run
Budget: 4 iterations remaining

It does not need every previous API response.

This is context engineering applied to runtime state.

Do not lose observations inside chat text¶

A tool result such as:

{
  "suite": "payment-regression",
  "passed": 418,
  "failed": 1,
  "failure": "RefundRetryTest"
}

can be stored as a normalized observation:

{
  "type": "TEST_RESULT",
  "source": "test_runner",
  "timestamp": "...",
  "data": {
    "suite": "payment-regression",
    "failed_tests": ["RefundRetryTest"]
  }
}

Conversation transcripts can remain useful audit/debug artifacts, but they should not be the only state store.

Intermediate artifacts¶

Agentic tasks often produce artifacts such as:

patches,
generated files,
query results,
report drafts,
candidate plans,
test logs,
screenshots,
deployment manifests.

Store large artifacts by reference:

{
  "artifact_id": "artifact-92",
  "type": "PATCH",
  "location": "object://runs/run-1842/patch.diff",
  "sha256": "..."
}

rather than copying the full payload into every subsequent context.

Checkpoints¶

Longer runs should not live only in process memory.

step completes
     ↓
state transition
     ↓
persist checkpoint
     ↓
next step

If execution is interrupted:

worker dies
network fails
process restarts

then:

load checkpoint
 ↓
revalidate environment
 ↓
resume

not:

read old chat transcript
 ↓
ask model what probably happened

Resume is not just continue¶

The checkpoint may be correct while the outside world has changed.

Example:

checkpoint:
PR #184 is open

30 minutes later:
PR was merged by a human

Before resuming, refresh relevant observations.

restore internal state
      +
refresh external preconditions
      ↓
continue / replan / stop

State versioning and concurrency¶

Multiple workers or events may update the same run concurrently.

Example:

worker A reads version 27
worker B reads version 27
A writes version 28
B also writes based on stale version 27

Useful techniques include:

optimistic locking,
compare-and-swap versions,
single-writer orchestration,
transactional updates,
event ordering.

For example:

UPDATE run
SET state=?, version=28
WHERE run_id=? AND version=27

If no row is updated, reload the current state.

Agent runtimes face the same concurrency problems as ordinary distributed applications.

Event log vs snapshot¶

Two common representations are useful.

Snapshot¶

current state object

Simple to read.

Event log¶

RUN_CREATED
TOOL_CALLED
TOOL_SUCCEEDED
PLAN_REVISED
APPROVAL_REQUESTED
APPROVAL_GRANTED
RUN_COMPLETED

Useful for audit and debugging.

Often the practical solution is both:

append events
    ↓
maintain current snapshot

Full event sourcing is unnecessary unless requirements justify it.

Scratchpad as explicit working state¶

A scratchpad may look like:

{
  "hypotheses": [
    {"text": "shared fixture race", "status": "LIKELY"},
    {"text": "retry timer bug", "status": "REJECTED"}
  ],
  "files_to_inspect": ["RetryService.java", "RefundRetryTest.java"]
}

This is more useful operationally than a long prose reasoning history.

The runtime needs relevant intermediate conclusions, evidence, and open questions; it does not need to persist a hidden reasoning transcript as canonical state.

Derived state vs source state¶

For example:

raw observations:
- error rate 12%
- deployment 5 minutes ago

hypothesis:
- deployment likely caused regression

Represent these separately:

FACT / OBSERVATION
vs
HYPOTHESIS / DERIVED

This reduces the chance that a previous model assumption becomes a “fact” in later context.

State cleanup and retention¶

Not every run artifact needs permanent retention.

Different data classes can have different policies:

run metadata: 90 days
sensitive tool payload: 7 days
large artifacts: 30 days
aggregated metrics: longer

This matters for privacy, security, and cost.

Anti-pattern: chat history as a database¶

Weak design:

messages[]

where important state exists only somewhere in prose.

Problems:

difficult deterministic querying,
difficult resume,
stale information remains visible,
context keeps growing,
concurrency is hard to manage,
state is implicit.

Better:

structured canonical state
+ event/trace history
+ selected context projection

Anti-pattern: copying every observation into every prompt¶

After 50 steps, this creates token and noise explosion.

Prefer:

persist full evidence externally
        ↓
select / summarize / reference relevant evidence
        ↓
current model call

Takeaways¶

Canonical execution state is runtime-owned, not model-owned.
Model context is only a relevant projection of that state.
State, context, scratchpad, and long-term memory are different concepts.
Run status, budgets, approvals, completed steps, and side-effect metadata should be explicit structured state.
Checkpoint/resume is a core reliability capability for long or asynchronous runs.
Revalidate the external environment after resuming.
Concurrency and state versioning remain normal software-engineering problems.
Represent observations and hypotheses separately.
Store artifacts by reference instead of putting full payloads into every context.
Chat history can be useful trace data, but it is a poor canonical state store.