Skip to content

09 – Tool Calling

Tool calling is the mechanism that lets an LLM request an action or external capability without directly executing that action itself.

The key mental model is:

LLM decides what it wants to do
        ↓
structured tool call
        ↓
application validates the request
        ↓
application executes real code
        ↓
tool result goes back to the model
        ↓
model continues with the new information

The LLM is not the executor. It is a decision-making component that can propose a tool invocation.

Why tool calling exists

An LLM by itself cannot reliably access current databases, send emails, query private systems, modify files, call internal APIs, or perform deterministic calculations unless those capabilities are exposed to it.

Tools bridge that gap.

Examples:

  • get_customer(customer_id)
  • search_documents(query)
  • create_ticket(title, priority)
  • get_weather(location)
  • run_build(branch)
  • calculate_tax(amount, country)

The model receives descriptions and schemas for the available tools and can choose one when the current task requires it.

Tool definition as a contract

A tool should have a clear machine-readable contract.

Conceptually:

{
  "name": "get_order",
  "description": "Retrieve an order by its identifier",
  "parameters": {
    "type": "object",
    "properties": {
      "order_id": {
        "type": "string"
      }
    },
    "required": ["order_id"]
  }
}

The important part is not the exact syntax. The important part is that the application defines:

  • the tool name,
  • what it does,
  • what arguments it accepts,
  • which arguments are required,
  • what types are expected.

This is similar to defining an API contract or typed function signature.

Example: order lookup

User:

Where is order ORD-12345?

The LLM cannot know the current order state from training data.

It can request:

{
  "tool": "get_order",
  "arguments": {
    "order_id": "ORD-12345"
  }
}

The application then executes something equivalent to:

order = order_service.get_order("ORD-12345")

and returns a result:

{
  "order_id": "ORD-12345",
  "status": "shipped",
  "carrier": "DHL",
  "estimated_delivery": "2026-09-01"
}

The model can now answer:

Your order has shipped with DHL and is currently estimated to arrive on September 1.

The current fact came from the tool, not from the model's training knowledge.

Tool calling is not direct function execution

A useful separation is:

LLM layer
- choose tool
- generate arguments
- interpret tool result

Application layer
- authenticate
- authorize
- validate
- execute
- handle errors
- audit

A dangerous architecture would effectively do:

LLM says "delete user 123"
        ↓
execute immediately

A safer architecture is:

LLM proposes delete_user(user_id=123)
        ↓
validate schema
        ↓
authenticate caller
        ↓
check permission
        ↓
check business rules
        ↓
possibly require confirmation
        ↓
execute

The model's tool choice is an input to application logic, not authorization.

Read tools versus write tools

Not all tools have the same risk.

Read-only tool

get_customer
search_docs
get_invoice

Failure usually means incorrect or missing information.

Mutating tool

send_email
cancel_order
restart_server
transfer_money

Failure can create external side effects.

For mutating tools, the application usually needs stronger controls:

  • authorization,
  • confirmation,
  • idempotency,
  • auditing,
  • retry policy,
  • rate limits.

Tool results are context

A tool result normally becomes additional model context.

User request
    ↓
LLM
    ↓
tool call
    ↓
tool execution
    ↓
tool result
    ↓
LLM
    ↓
final answer or another tool call

This is the basic building block of an agentic loop.

Tool calling itself is not necessarily an agent. A single controlled tool call can be a deterministic application workflow.

One tool call versus a loop

Simple flow:

question
  ↓
LLM
  ↓
get_order
  ↓
answer

Agent-like flow:

goal
 ↓
LLM
 ↓
search_repository
 ↓
LLM
 ↓
read_file
 ↓
LLM
 ↓
run_tests
 ↓
LLM
 ↓
edit_file
 ↓
LLM
 ↓
final result

The second example repeatedly observes results and decides the next action. That repeated decision/execution cycle is where agentic loops begin.

Tool schema design matters

Bad tool:

execute(action: string, payload: object)

This gives the model a very broad and ambiguous interface.

Better:

get_customer(customer_id)
create_support_ticket(title, description, priority)
cancel_order(order_id, reason)

Narrow tools are easier to:

  • understand,
  • authorize,
  • validate,
  • test,
  • observe,
  • restrict.

This follows normal API design principles.

Prefer semantic tools over infrastructure tools

Suppose the task is to refund an order.

Weak abstraction:

execute_sql(sql)

Better abstraction:

refund_order(order_id, reason)

The second tool preserves domain boundaries and lets deterministic application code enforce the real refund rules.

A useful principle:

Expose business capabilities to the model, not unrestricted infrastructure primitives, unless unrestricted access is intentionally required and strongly sandboxed.

Tool argument validation

Even if the platform guarantees schema-compatible output, business validation is still necessary.

Tool call:

{
  "tool": "transfer_credit",
  "arguments": {
    "customer_id": "C-123",
    "amount": 1000000
  }
}

The arguments may be structurally valid while the amount violates policy.

Therefore:

schema validation
       ↓
business validation
       ↓
authorization
       ↓
execution

Error handling

Tools fail like normal software dependencies.

Possible failures:

  • timeout,
  • HTTP 500,
  • invalid arguments,
  • permission denied,
  • resource not found,
  • rate limit,
  • temporary outage.

The application should convert raw infrastructure failures into clear tool results where appropriate.

Example:

{
  "status": "error",
  "error_code": "ORDER_NOT_FOUND",
  "message": "No order exists with this identifier"
}

The model can then decide whether to ask the user for another ID instead of hallucinating an order.

Do not hide important errors from the model

Bad flow:

tool fails
 ↓
application returns empty object
 ↓
LLM guesses what happened

Better:

tool fails
 ↓
structured failure result
 ↓
LLM understands the failure
 ↓
retry / ask user / stop

Example: support assistant

Available tools:

get_customer(email)
get_recent_orders(customer_id)
create_support_ticket(customer_id, category, summary)

User:

My last order never arrived. Please open a ticket.

Possible sequence:

1. LLM → get_customer(email)
2. Application validates and executes
3. Result → customer_id C-847
4. LLM → get_recent_orders(C-847)
5. Result → last order ORD-551, status shipped
6. LLM → create_support_ticket(...)
7. Application checks authorization and executes
8. Result → ticket SUP-9182
9. LLM tells the user the ticket number

The model coordinates the workflow, but every real side effect remains under application control.

Tool calling versus RAG

They solve different problems.

RAG usually means:

retrieve relevant knowledge
        ↓
put it into context
        ↓
answer

Tool calling means:

choose an external capability
        ↓
execute it
        ↓
observe the result

A retrieval system can itself be exposed as a tool, but conceptually retrieval and action are separate concerns.

Tool calling versus MCP

Tool calling is the model/application interaction pattern.

MCP is one possible standardized way to expose tools and other contextual resources to AI clients.

Conceptually:

Tool calling = how a model requests capability execution

MCP = a protocol that can provide those capabilities to AI applications

Do not treat them as competing concepts.

Testing tool-enabled systems

Test at several levels.

Schema tests

Does the tool reject malformed arguments?

Authorization tests

Can unauthorized users invoke a protected action?

Execution tests

Does the underlying tool implementation work correctly?

Model behavior tests

Does the model choose the correct tool for representative requests?

End-to-end tests

Does the full loop produce the expected result without unintended side effects?

Practical design rules

  • Keep tools narrow and semantically meaningful.
  • Treat tool calls as untrusted requests until validated.
  • Keep authorization in deterministic application code.
  • Separate read-only and mutating capabilities.
  • Return explicit structured failures.
  • Design mutating tools for retries and idempotency.
  • Log tool name, arguments where safe, result status, latency, and caller context.
  • Do not give the model broader permissions than the task needs.

Mental model

LLM = planner / chooser
Tool = capability contract
Application = policy + executor
External system = real world state

The important boundary is:

The LLM can decide what it would like to do. The application decides whether and how that action is allowed to happen.