10 – Reliability and Deterministic Boundaries¶

LLM-based systems combine two different worlds:

probabilistic model behavior
          +
deterministic software behavior

Reliable AI applications work well when those boundaries are explicit.

The core rule is:

Use the LLM for interpretation, generation, ranking, and flexible reasoning. Use deterministic software for contracts, permissions, invariants, money, identity, persistence, and side effects.

Why this matters¶

A normal function is expected to follow its code path exactly.

result = calculate_vat(1000, 0.27)

Given the same inputs and code, the application expects the same result.

An LLM call is different:

same prompt
same model
same context
    ↓
possibly different wording or decision

Even when model behavior is very stable, treating it as perfectly deterministic creates fragile systems.

Decide what the model is allowed to decide¶

A useful architecture question is:

What requires judgment?
What requires guarantees?

Example: support-ticket routing.

Reasonable LLM responsibility:

classify message as:
- billing
- technical
- account
- other

Deterministic responsibility:

if category == "billing":
    assign billing queue

The LLM interprets language. The application performs the actual routing rule.

Do not encode critical invariants only in prompts¶

Weak design:

System prompt:
"Never approve refunds above 500 EUR."

This may help model behavior, but it should not be the real enforcement layer.

Better:

if refund.amount > 500:
    require_manager_approval()

The prompt can explain policy to the model, but deterministic code enforces it.

Structured output is a boundary¶

Suppose the model classifies a request.

Instead of:

"This seems quite urgent and probably related to billing."

prefer a contract such as:

{
  "category": "billing",
  "priority": "high"
}

Then validate it before using it.

LLM
 ↓
structured result
 ↓
schema validation
 ↓
business validation
 ↓
application logic

This limits the amount of free-form model behavior entering the deterministic core.

Validation has multiple layers¶

Consider:

{
  "currency": "EUR",
  "amount": 1000000
}

It may be valid JSON and valid against a schema.

But business logic can still reject it.

Think in layers:

1. Syntax / transport validity
2. Schema / type validity
3. Business-rule validity
4. Authorization validity
5. Side-effect execution

Do not collapse these into one prompt.

Retries¶

Retries are useful for transient failures, but dangerous when used blindly.

Possible retryable failures:

model timeout,
provider 5xx,
temporary rate limit,
network error,
temporary tool outage.

Not automatically retryable:

invalid user input,
permission denied,
business-rule violation,
repeated malformed logic caused by a bad prompt,
a non-idempotent side effect that may already have succeeded.

A useful retry policy distinguishes failure classes.

failure
  ↓
classify
  ├── transient → retry
  ├── permanent → stop
  └── ambiguous side effect → reconcile before retry

Idempotency¶

Idempotency becomes essential when agents or tool-enabled systems can repeat actions.

Imagine:

LLM → create_payment(100 EUR)
network timeout
application does not know whether payment succeeded
retry

Without idempotency, the customer might be charged twice.

A safer pattern:

create_payment(
    amount=100,
    idempotency_key="order-123-payment"
)

The external system can recognize the repeated request as the same logical operation.

This is not specifically an AI principle. AI systems simply make it more important because retries and loops are common.

Timeouts and budgets¶

Every model or tool call needs limits.

Examples:

request timeout,
maximum tool-call count,
maximum loop iterations,
maximum total elapsed time,
maximum token usage,
maximum cost per task.

Without explicit limits:

agent
 ↓
retry
 ↓
search
 ↓
retry
 ↓
tool
 ↓
search
 ↓
...

can become an unbounded process.

Stop conditions¶

Agentic workflows need deterministic stop conditions around probabilistic reasoning.

Examples:

stop if task completed
stop if max_steps == 10
stop if cost > budget
stop if tool returns terminal error
stop if human approval is required

Do not rely only on the model deciding that it has done enough.

Fallbacks¶

Fallbacks can exist at several levels.

Model fallback¶

primary model unavailable
       ↓
secondary model

Behavior fallback¶

AI classification confidence/evaluation check fails
       ↓
manual queue

Tool fallback¶

semantic search unavailable
       ↓
keyword search

Product fallback¶

AI answer cannot be supported safely
       ↓
show source documents or escalate to human

A fallback should be intentionally designed, not random emergency code.

Confidence is tricky¶

Do not assume that a model-generated number such as:

{
  "confidence": 0.98
}

is a calibrated probability.

The model can produce a confidence field because you asked for one, but that does not automatically mean requests labeled 0.98 are correct 98% of the time.

If confidence matters, calibrate it empirically using evaluation data or derive confidence from more reliable signals.

Example: invoice extraction¶

Goal: extract invoice data from uploaded text.

Model output:

{
  "invoice_number": "INV-2026-991",
  "currency": "EUR",
  "total": 1250.50
}

Reliable flow:

uploaded document
      ↓
LLM extraction
      ↓
schema validation
      ↓
check currency against allowed list
      ↓
check total >= 0
      ↓
check invoice number uniqueness
      ↓
possibly verify against source text
      ↓
persist domain object

The model performs fuzzy extraction. The application protects invariants.

Example: code-generation agent¶

The agent proposes a code change.

Weak flow:

LLM writes code
 ↓
merge

Better flow:

LLM writes code
 ↓
formatter
 ↓
compiler / type checker
 ↓
unit tests
 ↓
integration tests
 ↓
static analysis
 ↓
review / policy checks
 ↓
merge

The more deterministic verification you can put after generation, the more safely you can automate.

This is one of the strongest patterns in AI-assisted software engineering:

Let the model generate candidates. Let deterministic systems verify what can be verified.

Failure recovery¶

Reliable systems make failure state explicit.

Instead of:

something failed → ask model to try something else

track structured state:

{
  "step": "create_ticket",
  "attempt": 2,
  "status": "failed",
  "error_code": "RATE_LIMIT",
  "retryable": true
}

This lets orchestration logic, not just model intuition, determine what happens next.

Observability¶

For production systems, record enough data to understand failures and cost without leaking sensitive content.

Useful signals:

model name/version,
prompt/template version,
request latency,
input/output token counts,
tool calls,
tool latency,
retry count,
finish reason,
validation failures,
evaluation metrics,
total task cost.

Without observability, AI reliability problems often look like random anecdotes.

Reliability is system-level¶

Do not ask only:

"How accurate is the model?"

Ask:

What happens when the model is wrong?

A model with imperfect accuracy can support a highly reliable application if mistakes are bounded, detected, validated, or safely recoverable.

A stronger model does not remove the need for good system design.

Practical boundaries¶

Good LLM candidates:

summarize text,
classify ambiguous language,
extract fuzzy information,
generate drafts,
rank candidates,
propose plans,
interpret user intent.

Good deterministic-code candidates:

authorization,
monetary calculations,
schema validation,
database constraints,
retry policy,
workflow limits,
rate limiting,
state transitions,
final safety checks.

Mental model¶

Probabilistic intelligence
        ↓
controlled interface
        ↓
deterministic guardrails
        ↓
real side effects

The goal is not to make the LLM deterministic.

The goal is to design a system where probabilistic behavior is used where it creates value and deterministic boundaries protect everything that requires guarantees.