OpenAI function calling: why calls fail in production
OpenAI function calling turns the model from a text generator into an executor: instead of answering in words, it returns a structured tool call that your code executes.
In a demo it looks reliable, but in production the AI assistant integration breaks in several predictable places.
The first source of failures is invalid model arguments. The same field arrives sometimes as a string, sometimes as a number, dates come in free format, nested objects lose some keys, and sometimes the model confidently inserts a value that is not in your reference.
Without strict input validation, such a call goes into business logic and fails there, leaving data half-assembled.
The second is a mismatch between descriptions and the actual contract. Tool descriptions in LLM tool calls live separately from the code: a parameter was renamed in the service, but the instruction remained old, and the model continues to assemble the old set of fields.
Same with optional parameters — if the field is not marked required, the model simply will not send it.
The third is the lack of retries and handling of parallel calls. A response arrives with a rate limit or timeout, the call is silently lost, and the user sees that the assistant “forgot” to perform the action.
When the model returns several calls at once, some depend on the previous result — and without explicit ordering and duplicate protection, a retry becomes a duplicate request or payment.
We analyze each scenario at the level of the argument schema, instruction text, and retry logic, so a failure remains a log line, not a client incident.
What managed function calling integration gives
When the model chooses the tool itself, scenario quality rests not on prompt wording, but on the strictness of the call description. Managed call integration is a contract: which functions are available, which arguments are required, what to do on type mismatch. The more precisely it is defined, the fewer decisions the model makes at random.
We describe schemas declaratively and validate each argument before execution — at this step a type-enforcing layer like Pydantic works.
A type error or missing field is cut off at input, not inside business logic, so one tool failure does not unfold into a chain of wrong actions.
We count the share of successful calls per tool separately: how many calls reached the result, how many went to retry, how many the model discarded itself. This shows where the schema is ambiguous — for example, two tools compete for the same request.
Independent calls go in parallel, so assistant response time is determined by the slowest of them, not the sum of all. A limit on chain depth and number of retries prevents the scenario from looping when an external service responds slowly.
The retry layer with idempotency covers network failures, and a strict response format covers parsing errors. This is failure class reduction: the scenario does not fail entirely, but degrades predictably, preserving a clear user response.
Support requests decrease not because of polite wording, but because the user more often completes the action to the end. AI scenario stability is confirmed by call logs and metrics, not by the feeling that “it got better” — on them we also build regression runs before changes.
Tool call schemas: which approaches to choose
A tool call schema is a contract between the model and the service: which functions are available, which arguments they accept, and what they return.
How precisely the contract is described and who holds control at what moment determines the number of dialog iterations and the place where validation cuts in.
Assistant operating modes differ exactly by this — who owns control at the call moment. We select the mode for the task, not take one default for the entire project.
| Mode | When suitable | Limitations |
|---|---|---|
| Single function call | Steps depend on each other: the previous result is needed as the next argument | Each step is a separate iteration, response time grows |
| Parallel calls | Independent operations: order status, stock balance, delivery time — in one pass | Idempotency required; some calls may fail, response order not guaranteed |
| Returning control to the service side | Complex logic: branching, approvals, retries, access to closed data | The decision is made by the model, while orchestration and access rights remain with the service |
Integration limitations often lie not in the model, but in the contract.
Arguments arrive structured, but without validation by the described model extra fields and wrong types are possible — schema validation cuts off such calls before execution and returns a clear error to the model.
For parallel branches we fix the tool timeout and behavior on partial success: retry, compensation, or explicit refusal.
How we implement integration: stages of engineering work
Function calling turns the assistant from a text generator into an executor: the model returns not a paragraph, but a structured tool call with arguments.
Therefore integration implementation stages are built around contracts — with a vague schema the assistant calls the wrong tool or substitutes the wrong parameter.
- Audit of assistant scenarios. We review recorded dialogs and regulations, write out the actions the assistant initiates itself and those requiring confirmation. The output is a scenario map with frequency and cost of error in each.
- Setting up tool schemas. We describe each tool strictly: required fields, allowed values, number bounds, model-readable wording. We keep schemas in Pydantic models so the contract and code do not diverge.
- Call modes. We mark where parallel calls are appropriate (checking order and stock simultaneously), and where strict sequence is needed. Separately we fix tools the assistant must not call.
- Call validation on staging. We run recorded and synthetic dialogs, count the share of correct calls, false positives, and retries. We deliberately break schemas: the assistant must clarify, not invent an argument.
- Failure handling and observability. We configure response validation, retries for safe operations, and instant refusal for charges and sends. We log the “request — call — result” bundle so growth of invalid schemas is visible before complaints.
- Project handover. We hand over schemas, tests, the regulation for editing descriptions, and the release order for changes, and train the customer team. Further contract edits go through the same staging.
Configurations, schemas and regulations: what remains in the project
After implementation, the project retains not only a working model call, but also a set of artifacts your team can use to develop the solution without us.
We collect them in one handover and go through the list together with your engineer at acceptance — so every setting is explained, not just copied.
- Integration documentation — the request flow from your service to the model and back, logging points, behavior when the external service is unavailable and when rate limits are exhausted.
- Tool descriptions — the purpose of each function, composition and types of arguments, required and optional fields, examples of valid and boundary values: by these descriptions the model chooses what to call, so they are analyzed in more detail than the rest.
- Data models — Pydantic structures that validate arguments before the call and parse responses: an incorrect field set is cut off on our side, not sent to the service.
- Access policy to services — which keys and permission scopes each tool needs, where they are stored, how they are rotated, and what to do on suspicion of a leak.
- Schema update regulation — the order of adding a new tool or changing an existing one, versioning rules for descriptions, and a way to roll back to the previous version if behavior changed.
- Validation log — a set of scenarios for parallel calls, service failures, incomplete and invalid arguments, with run results before handover: it is visible which paths were tested and which remained outside coverage.
- Team instructions — how to deploy the environment, where to change limits and response timeout, where to look when responses diverge between environments.
- Handover of settings — the list of environment variables, service bindings, and operating modes with an explanation of why each mode was chosen and what changes when it is switched.
The set remains in your repository and is updated with the code — a new function without a description and a regulation entry simply will not pass validation.
Case: how we reduced the share of call failures sixfold
An online store support service: the assistant called external functions — order status check, delivery cost recalculation, ticket creation.
Argument schemas were described in words in the system instruction, and the model returned them in free form: strings instead of numbers, extra fields, calls to functions that do not exist in the service.
The share of call failures held at 18% — every fifth dialog went to an operator for manual review, and the cause could not always be reconstructed from logs.
What changed
Arguments were moved to strict schemas: the contract of each function is described by a data model with types and required fields, and a call without validation does not go further.
Parallel calls replaced sequential ones where functions are independent — order status and delivery recalculation are requested in one step, not two in a row.
We added retries with correction: if an argument fails validation, the error text is returned to the model, and it fixes the call itself. Validation moved to our service, before accessing external systems, so an invalid call no longer reaches the database.
What result
All calls go through the function calling OpenAI mechanism with hard-defined schemas. The share of call failures fell from 18% to 3% — six times fewer. Operator load reduced from 21% of dialogs to 5%, and response time decreased from 6 to 2.5 seconds due to parallel calls.
Implementation took three weeks: rewriting contracts, a new validation layer, and a run on historical dialogs before launch.
How strict schema differs from free response parsing
Free response parsing is when the application extracts data from text with regexes and substring search, and the model is asked to “return structure without explanations”.
It passes in a demo, but falls apart in work: an introductory phrase is added, a field is renamed — and the parser silently returns an empty value instead of an error.
A strict argument schema translates the agreement into a contract: fields, types, requiredness, allowed values. The OpenAI platform on a function call checks arguments against it and does not pass a nonconforming variant.
The code receives a structure ready for validation by a data model — not text that still needs interpretation.
The difference is visible in four things: what reaches the code, how the scenario survives an edit to the request wording, where a format error goes, and what support costs.
| Criterion | Strict schema | Free parsing |
|---|---|---|
| What reaches code | typed structure | text for substring search |
| Edit request to model | contract holds | parsing breaks silently |
| Invalid model response | rejected before function call | arrives as empty field |
| Support | edit schema description | set of regexes grows |
The boundary lies at the response consumer. If values go to a function call, a database write, or scenario branching, a schema is needed: free parsing there gives silent failures that are sought in logs. If the result is read by a human, parsing is tolerable — the error cost is limited to one message.
The schema is responsible for form, not meaning: value enums, ranges, and business rules are checked on our side.
A rejected call is returned to the model with the validation error text — it corrects the arguments and repeats the call, so the scenario converges to a correct result instead of failing.
Common technical doubts before integration
Before integration starts, the same questions come up: will the schema withstand growth in the number of tools, what to do with invalid arguments, how to react to changes on the model provider side. Below are short answers for each.
Several tools per response — manageable?
Parallel calls are a standard mode: we collect results by call IDs and return them as one message, so the response does not “fall apart” even if one tool responds longer than others. Where strict sequence is needed, we narrow the mode to one call per step.
What if the model sends invalid arguments?
Arguments are validated by a Pydantic model before the code call, so a missing field or wrong type does not reach business logic. We return the error text to the model and ask for a corrected call — most such cases close on the second attempt.
New tool — rewrite the integration?
Tools live in a registry: we describe the function and its model, and generate the schema for the model automatically. The dispatcher, logging, permission limiting, and retries connect to the new tool by common rules.
Provider changes model behavior — what then?
An adapter sits between our code and the API, so a model version change is fixed in one place. Contract tests on recorded responses catch discrepancies before release.
Let's discuss the task: what's needed to estimate integration
To return with an estimate and work plan, not general questions, we need a minimal set of project inputs. Below is a list of what determines the scope of Function Calling integration and the launch order.
If some points are not yet resolved, that's fine: we'll record them as open questions and propose a default option.
- Scenarios: which actions the model must perform itself, and which remain with a human — this sets the number of functions and the dialog branching logic.
- Call schemas: the list of functions, their arguments and required fields, value constraints. It is useful to see the current description — a draft JSON schema or Pydantic model.
- Models and modes: which models you use, whether parallel calls and streaming response are needed, what response you consider acceptable.
- Data sources and access: which systems are called from functions, which connection methods are allowed on your side, where keys are stored.
- Environment and security: where the solution is hosted, logging and dialog storage requirements, who is responsible for access.
- Load and fault tolerance: expected number of requests, peak periods, share of requests with function calls, what is critical when an external system is unavailable.
- What already exists: work in progress, tests, known errors in argument parsing and retries — we'll save a diagnostic stage.
Describe the task in free form and attach what you have. After reviewing the inputs, we'll return with an estimate and work plan: stages, schema composition, risk areas, and launch order.
