LLM tool calling connects a language model to APIs, databases, and application functions, but a successful integration requires more than exposing a function name. This tutorial provides a reusable design for defining safe tools, validating arguments, controlling permissions, handling timeouts and retries, and observing every execution.
Overview
Tool calling, also known as function calling, lets an LLM return a structured request for an application-side function. The model does not normally execute that function itself. Instead, your application receives the requested tool name and arguments, checks whether the request is allowed, runs the function, and sends the result back to the model for a final response or another tool request.
A typical flow looks like this:
- Your application sends the user message and a list of available tools to the model.
- The model decides whether a tool is useful and returns a structured tool call.
- Your application validates the tool name and every argument.
- A controlled executor runs the function with the user’s permissions and an explicit timeout.
- The application records the result and sends a bounded version back to the model.
- The model either requests another permitted tool or produces a user-facing answer.
This separation is important. The model can suggest an action, but your application should remain responsible for authorization, validation, side effects, and failure handling. Treat tool calling as an API integration problem with an LLM interface, not as a replacement for normal application controls.
Tool design also affects prompt reliability. A narrow function with clear inputs is easier for a model to select correctly than a broad function that accepts arbitrary instructions. For related guidance on reliable structured responses, see Structured JSON Output from LLMs.
Template structure
A reusable tool definition should answer five questions: what the tool does, when it should be used, which arguments it accepts, what limits apply, and what the application returns. The exact schema format varies by AI API, but the design principles are portable.
1. Give the tool a narrow purpose
Use a name that describes one operation, such as get_order_status, search_knowledge_base, or create_calendar_draft. Avoid a generic tool such as run_action that hides several unrelated capabilities behind one interface.
2. Define explicit arguments
Each argument should have a type, a description, and sensible constraints. Specify required fields, allowed enumerations, maximum lengths, numeric ranges, and accepted formats. If an identifier must be a UUID or an email address, validate that in application code rather than relying only on the description.
{
"name": "get_order_status",
"description": "Retrieve the status of one order for the authenticated customer.",
"parameters": {
"type": "object",
"properties": {
"order_id": {
"type": "string",
"description": "The order identifier supplied by the customer."
}
},
"required": ["order_id"],
"additionalProperties": false
}
}The example is intentionally small. A tool should expose only the inputs needed for its task. Do not pass an entire user message to a database query or shell command when a validated identifier or enum will do.
3. Add an authorization boundary
Authorization belongs in the executor, not in the model’s instructions. Before running a tool, establish which user, tenant, role, or service identity the request is associated with. Check that identity against the requested resource and operation. A model-generated argument is untrusted input, even when it appears well formed.
4. Define a result contract
Return predictable results such as a status, a small data object, and an error category. Keep internal stack traces, credentials, private records, and unnecessary fields out of the model-visible result. A concise result is easier for the model to interpret and reduces context usage.
5. Specify failure behavior
Decide what happens when a tool is unavailable, arguments are invalid, a record is missing, or an operation requires confirmation. Use stable error categories such as validation_error, not_found, permission_denied, timeout, and temporary_failure. This gives the model and your application a consistent way to recover without exposing implementation details.
How to customize
Start with the smallest useful workflow, then add safeguards based on observed failures. The following checklist works for customer support tools, internal assistants, RAG applications, and AI workflow automation.
Separate read operations from side effects
Reading information and changing information should not have the same permission model. A search or lookup tool may be safe to run automatically, while sending an email, deleting a record, issuing a refund, or changing access should require stronger checks. For consequential actions, use a two-stage pattern: create a draft or preview first, then ask for explicit confirmation before committing.
Validate twice when the risk justifies it
Validate the structure immediately after parsing the model response. Validate business rules again inside the function or service that performs the operation. For example, a date may have the correct format but still be outside an allowed booking window. Defense in depth is especially useful when tools can modify data.
Control multi-step execution
Do not allow an open-ended loop of tool calls. Set a maximum number of steps, a total time budget, and a maximum response or result size. Store a trace of each call so that a failed workflow can be reconstructed. If the model repeatedly requests the same failed action, stop and return a clear recovery message instead of retrying indefinitely.
Use careful timeout and retry policies
Every network-backed tool should have a deadline. Retry only failures that are likely to be temporary, and use backoff rather than immediate repeated requests. Never blindly retry a non-idempotent operation such as a purchase or message send unless the downstream API supports an idempotency key. A timeout should produce a safe, user-understandable result and an internal diagnostic event.
Design for prompt injection
Retrieved documents, web pages, emails, and user-provided files can contain instructions that conflict with your application’s rules. Treat their contents as data, not authority. Keep system-level policies and tool permissions outside retrieved text, restrict tools to the current task, and require confirmation for sensitive actions. A RAG workflow should never gain permission merely because a retrieved document tells it to use a tool.
If your application includes retrieval, the RAG application development guide provides a useful companion for chunking, citations, and retrieval design.
Measure the workflow, not just the final answer
Track tool-selection accuracy, argument-validation failures, authorization denials, latency, retry counts, execution errors, and successful task completion. Review traces where the model selected the wrong tool or used a plausible but incorrect identifier. These signals are often more actionable than judging the final text alone. A prompt testing framework can help you run the same tool scenarios after changes to models, schemas, or instructions.
Examples
Example: a read-only knowledge search
Suppose an assistant can search internal documentation. The tool might accept a query, product area, and result limit. The executor should cap the limit, apply the current user’s access scope, and return document identifiers, short excerpts, and source metadata. It should not return an entire document collection to the model. If no results are found, return a distinct result rather than encouraging the model to invent an answer.
Example: creating a support ticket
A ticket tool could accept a title, description, priority, and category. The application should restrict priority and category to known values, limit text length, attach the authenticated user automatically, and remove unsupported fields. Before creation, the assistant can show a preview containing the title, category, and description. The user then confirms the action, and the executor creates the ticket with an idempotency key.
Example: handling a failed calendar lookup
If a calendar provider times out, the tool result might contain status: "temporary_failure" and a short message such as “The calendar service did not respond before the deadline.” The assistant should not claim that a meeting is available. It can explain that the check could not be completed and offer to try again. Internal logs should retain the provider error, request identifier, elapsed time, and retry decision without exposing those details unnecessarily.
For each example, test both the normal path and adversarial inputs: missing arguments, extra properties, unauthorized identifiers, oversized text, repeated calls, ambiguous requests, and provider failures. Include these cases in automated evaluations alongside normal success cases. See Prompt Testing Frameworks for a broader approach to repeatable LLM evaluations.
When to update
Review tool definitions whenever the underlying API, data model, permission rules, model behavior, or user workflow changes. A schema that was safe for a read-only prototype may need substantial revision when write operations are added. Revisit the design after a production incident, a new prompt-injection pattern, a change in authentication, or a migration to a different model or AI API.
At minimum, maintain a small regression set containing successful calls, malformed arguments, permission failures, timeouts, duplicate requests, and confirmation-required actions. Run it when you change tool descriptions, required fields, system prompts, retry logic, or result formatting. Compare not only final-answer quality but also call count, latency, error rate, and unauthorized-action rate.
To put this tutorial into practice, choose one low-risk read-only function. Write its schema, add strict application validation, enforce authorization, set a timeout, and log a trace for every call. Then add one deliberately failing test for each boundary. Once that workflow is stable, introduce a second tool and explicit step limits before attempting more autonomous agent behavior. This incremental approach makes LLM function calling easier to test, explain, and maintain as your application evolves.
