Flow rate forecast diverges from actual: where ML breaks in oil and gas
A well flow rate forecast that diverges from the actual measurement by 20–40% costs more than not having one.
It is used to plan the field development regime, the sequence of well interventions, and the loading of surface infrastructure — and the decision is cancelled a month later, when the actual measurement arrives, and the well stock has already been switched to a different operating mode.
The first source of divergence is telemetry quality. Wellhead sensors drift, some channels record measurements at different intervals, and periods of well stock downtime enter the history as zeros on par with a real pressure drop.
The model does not distinguish these states unless they are labeled, and confidently reproduces the artifact: during training the metric is good, in the new month the actual diverges.
The second source is task formulation. Reservoir water cut grows nonlinearly and differently within a single area, while the forecast is built with one model for the entire well stock, mixing wells with different histories and different pressure maintenance systems.
Added to this are inconsistent reference data and undocumented shutdowns: the history is clean on the outside but heterogeneous inside — and the model error goes into the production plan.
The conclusion is the same: the problem is almost never in the algorithm, but in the fact that the data and task formulation are not brought to a state in which the forecast is verifiable at all.
We analyze how this is closed — from labeling regime operations and cleaning telemetry to separate models by zones and divergence control at each cycle.
What ML gives in oil and gas: forecasting, modes, monitoring
Three directions work toward one result — the well stock produces more at the same cost level. Below is what exactly changes and by which metrics this is visible.
Forecast accuracy for each well
The model learns from telemetry history, geophysics, and repair logs and builds a flow rate forecast for an individual well, not for the entire field.
Over a horizon from a month to a quarter, the average forecast error drops by several times compared to linear extrapolation, so the divergence between plan and actual is visible in advance — there is time to redistribute production, rather than explain the shortfall after the fact.
Well operating mode selection
The second task is not to predict but to select setpoints: production rate, bottomhole pressure, pump mode. The search over options runs on a fast approximate model, the best solutions are checked against history and only then go into operation.
The result is an increase in cumulative production on the existing well stock and a reduction in unit lifting costs.
Early warning of water cut and well stock monitoring
Water cut is visible earlier by a set of signs: water cut dynamics, deviation of bottomhole pressure, growth of energy consumption for lifting. The signal comes weeks before water breakthrough, and the well is managed to be switched to a different mode or isolation is performed.
Then — continuous monitoring of the well stock: the model recalculates the state of wells, filters out false alarms, and gives the operator a short list of those who need intervention, instead of manually reviewing hundreds of charts.
Implementation options: point model, industry model, own environment
The choice of implementation option for predictive analytics is determined by three things: how much own history has been accumulated for the well stock, whether there is a requirement to keep data within the perimeter, and what the cost of error is — a well shutdown and a false stop cost differently. Below are four approaches that are actually chosen between in industry projects.
They have one thing in common: the model is not launched "as is".
First, telemetry, pump operating modes, field measurements, and geophysics are brought to a unified form according to the industry standard OSDU, and only then is the model fine-tuned on the customer's historical data.
What differs is the amount of calibration, the deployment location, and who is responsible for maintenance.
Comparison of approaches
| Option | When it fits | Limitations |
|---|---|---|
| Point model for one task | One area or equipment type, history is already labeled, a fast verifiable result is needed | Does not transfer to other formations: on a new well stock quality drops and training starts over |
| Industry forecast model | Not enough own history, well modes are typical, forecast is needed for depleted areas | Requires calibration on local data, sensitive to atypical reservoir conditions |
| Hybrid model with reservoir physics | High cost of error, heterogeneous reservoir, development specialists are in the loop | Longer rollout: requires equations of state and validation with field services |
| Deployment on customer's servers | Data does not leave the perimeter, there are regulator requirements and an in-house IT department | The customer maintains infrastructure and model updates, a monitoring duty officer is needed |
Workflow: data audit, design, tuning, verification, handover
We build the workflow so that by the time the model is tuned, the data, target metrics, and verification method are fixed — otherwise the forecast quality can be neither confirmed nor improved. The stages are the same for any object: a well, a pad, or an injection area.
-
Source data audit. We review daily reports, unit telemetry, injection and well treatment history, and geophysical surveys.
We check history depth, measurement frequency, gaps, time breaks, and discrepancies between sources — at the output we get a list of features suitable for forecasting.
-
Solution design and target metrics. We fix the forecast object (flow rate, water cut, pump equipment failure risk), horizon, and allowable error. This way the result is accepted by a formal criterion, not by the impression from a demo.
-
Model tuning. We assemble data preparation and training into a reproducible process: the same extract gives the same result. Out of several approaches, we keep the one that is stable across different areas of the field.
-
Model verification on retrospective data. We run the model over past periods under conditions close to real: it sees only the data that were available at that time. We separately look at wells with a short history and areas where the operating mode changed.
-
Model maintenance procedure. We describe how to notice degradation: regular comparison of forecast with actual, control of the outlier share, retraining procedure when development modes change.
-
Handover of the solution to the customer. We deliver a working service, integration with the accounting system, documentation, and access to the infrastructure, and train your engineers. After that, support follows the agreed procedure.
What remains with the customer: models, schemes, procedures, control panel
We hand over not "a model that works on our laptop", but a kit according to which the system lives and develops with your team's efforts. Below is what physically remains with the customer after handover.
- Model documentation: what it forecasts, on which historical period it was trained, where the applicability boundaries are, how to read the confidence estimate, and what to do when input data goes beyond the training sample.
- Data flow scheme: the path from sensors and telemetry to the data mart, where each source is visible, the arrival frequency, check points, and places where corrupted or delayed measurements are discarded.
- Retraining procedure: list of triggers — shift in reservoir properties, metric degradation, introduction of a new well stock, — the order of verification on a holdout period, and conditions for rollback to the previous version.
- Metrics control panel: forecast quality for each well stock object, share of gaps, divergence of forecast from actual per shift and per month; the production engineer sees exactly where the model errs, not an average figure for the field.
- Data access rights: roles by areas and objects, separation of historical and operational data, log of access to sensitive datasets.
- Instructions for production engineer and dispatcher: how to interpret the signal, at which deviations to escalate, and when not to trust the model's decision.
- Version log: which configurations and models were in operation, what changed between them, and how to reproduce any past build.
Handover goes together with training your team: we go through incidents on real daily data, show how to retrain the model and not break the data mart. Describe the task — we will come back with an estimate and work plan.
Case study: water cut forecast returned wells to operation
On an asset with a well stock of several hundred wells, water cut grew faster than planned: wells went into downtime, and decisions on them were made based on the last flow rate measurement.
Reassessment of one well took up to two days, and up to half of well interventions were planned late — after the flow rate had already dropped.
We collected the history of telemetry, flow rate and injection measurements, repair and treatment logs, and took initial conditions for the formations from a model built in Petrel.
On these data, the model forecasts water cut and flow rate for a horizon of up to six months — separately for each formation and by pump operating mode, with retraining on fresh measurements.
What changed in the field
The deviation of forecast from actual over a three-month horizon decreased from 30–35% to 8–10%, and for wells with continuous telemetry — to 6%. This made it possible to bring 23 wells out of downtime and return them to the active well stock.
Planning of well interventions moved to a proactive mode: the calendar first includes wells with a predicted increase in water cut, not those where the decline is already visible from measurement.
Geologists received a ranked list of wells with an estimate of the effect from treatment and a deadline by which the intervention still gives a result.
The time for reassessment of one well decreased from two days to half an hour, and the share of well interventions confirmed by the forecast doubled.
How does flow rate forecasting differ from a well stock monitoring system?
These are different systems, and they are confused because both work with well data. Flow rate forecasting answers the question "what will happen": the model estimates production over a horizon from weeks to months based on the well stock's operating history.
Monitoring answers the question "what is happening now": it keeps the current state of the well stock in view and highlights what requires intervention.
The data requirements also differ. Forecasting needs a long comparable history for each well and correct labeling of shutdowns, and it is verified on retrospective data — whether the model hit the actual over past periods.
Monitoring lives on the telemetry stream: completeness of channels and resistance to gaps are important, and the cost of error is measured in hours to reaction, not in percentages over the horizon.
Monitoring shows a snapshot of the well stock: for each well, deviation from its own baseline is visible — bottomhole pressure drop, water cut growth, pump supply failure, mode misregulation.
Threshold notifications for wells are configured not by a single number for the entire well stock, but by the lifting method, mode, and history of the specific well: otherwise a stream of false alarms devalues the signal.
Diagnostics of deviations in operation is the next layer: it is important not only to see a parameter drift, but also to understand the cause and propose an action — change the mode, bring the well to stable operation, plan a repair.
Here the systems meet: monitoring supplies the actual and features, forecasting completes what the current trend will lead to.
Hence the practical conclusion. Forecasting without monitoring diverges from the well stock reality and remains a report, while monitoring without forecasting is reactive — it reports flow rate loss when it has already happened.
Common doubts of engineers: data, integration, maintenance
Engineers always have the same doubts before launch: what to do with gaps in the measurement history, how to integrate into the existing accounting loop, what will happen when the mode changes, and who is responsible for the model after handover. Below are answers for each point.
There are gaps and different polling intervals in the history — will the model train?
We distinguish a sensor gap and a shutdown: we interpolate the first with a quality flag, and mark the second as a separate mode. Training uses a mask, and we check quality on areas with complete history — the estimate is not inflated.
How does the solution integrate with accounting systems?
Through the historical database and standard gateways, by default read-only — we do not touch the accounting loop. Writing setpoints, if needed, goes through a separate agreed channel with an action log and operator confirmation.
What happens when the well operating mode changes?
A mode change is not a failure but a signal: control of input data drift and divergence of forecast from actual trigger fine-tuning on a fresh window. While accuracy is outside tolerance, the system gives a forecast marked with low confidence.
Who maintains the model after implementation?
We stay on maintenance: we monitor forecast error, retraining schedule, and integrity of input data. Once a quarter — a report on metrics; if degradation is below threshold, we investigate the cause and fix the features.
Let's discuss the task: we will come back with an estimate, design, and work plan
To make the estimate substantive rather than "depends on the data", it is enough to describe a few things in the request. The more precisely the forecast object and horizon are formulated, the fewer assumptions go into the plan.
We review the request, clarify using open data, and come back with a timeline estimate, design, and work stages.
- Forecast object and horizon: well flow rate, water cut, pump equipment failure, reservoir depletion.
- Class of critical error: missed failure or false alarm — model tuning depends on this.
- Usage mode: hint to the engineer, output to the dispatcher, or automatic loop with operator confirmation.
- Composition of source data: telemetry, repair and well killing history, logging, data from neighboring wells, polling frequency, and retrospective depth.
- Loop constraints: where data is stored, whether training outside the perimeter is allowed, which transmission channels are already accepted in the field.
- Acceptance criterion: on which wells and over what period we verify quality and with which current estimate we compare.
From this, a stage-by-stage work plan is assembled: data preparation and labeling, signal verification, first model with validation on a historical period, pilot operation on a limited group of wells, handover to the customer's team.
Describe the task in free form — we will come back with an estimate of implementation timelines, the source data composition for the start, and a stage-by-stage plan.
