Why cron tasks silently stop running
Periodic tasks stop running not with a bang, but silently.
The entry remains in the schedule, the scheduler wakes up reliably, but the command fails at the very first step — and no one knows about it until a manual check weeks later shows that the report was not generated and the backup was not taken.
The cause is almost always the environment: cron runs the task not from your session, but in a non-interactive shell with a reduced set of variables.
A different PATH — and the program cannot be found; a different working directory — and relative paths lead nowhere; unsubstituted access variables — and the script fails on authorization.
No one checks the return code, and the failure is discovered only from the superuser's mail, which no one reads.
The second source of silent failures is a missed run. If at the required moment the machine was rebooting, the data directory was unavailable, or the previous run had not yet finished, the schedule is not caught up: the moment passed — the task did not execute.
And when a scheduled task starts more often than the work itself takes, instances overlap and compete for the same files — externally everything is "green" until the result turns out to be overwritten.
The failure surfaces only during a manual check, when the recovery window is already closed and part of the data is lost irretrievably.
We break down how this is closed: by checking the return code, protecting against overlaps, and a run log that makes a missed run visible before a manager asks about it.
What managed execution of periodic tasks provides
While the schedule of periodic tasks lives in the duty engineer's head and in several crontabs on different machines, execution relies on human memory.
An update that was forgotten to be transferred drops out of the schedule, and it is noticed by the consequences — when data has not been updated or a report has not been sent to the recipient.
We convert tasks into managed units: each has a fixed schedule, entry point, and a prohibition on parallel execution on the same resource. Scripts become idempotent — a retry after a failure produces the same result, not duplicates.
The schedule is stored in a repository and checked before rollout — manual editing on the server goes out of practice.
Execution reliability rests on the return code and the execution log. Each run writes start, end, duration, and result to a log with rotation, and a non-zero return code goes to error notifications — to the on-duty shift chat, indicating the task and machine.
Failure detection time is reduced from hours to minutes: the failure is learned about not in the morning from a complaint, but at the moment of failure.
Then the effect accumulates: automation of routine operations removes repetitive actions — certificate renewal, cleanup, exports, log rotation — and the share of manual operations falls to isolated cases. The on-duty shift stops being a dispatcher of runs and does work where a human is truly needed.
After a month, predictability is visible: the same start time, a clear log for each task, no "why didn't it work." Reducing manual labor here is not an end in itself, but a consequence of execution becoming observable and reproducible.
Ways to run scheduled tasks: comparison of approaches
The choice of a scheduling method is determined not by taste, but by what should happen during a failure: a missed run, a duplicate task, or data divergence.
A simple crontab on a single node covers backups, log rotation, and reconciliations while the server is alone and the task does not move.
As soon as there are more nodes, and execution must survive a machine failure, limitations come to the fore. Below are real alternatives and the framework of each.
| Option | When it fits | Limitations |
|---|---|---|
| System cron on a node | single server, short scripts, routine operations | no history or retries on failure, the task does not move to another node, time is taken from the server's time zone |
| System service manager timer | start after a service starts, dependency on other units, resource limits | tied to a specific machine, schedule syntax does not transfer from crontab one-to-one |
| Scheduler inside the application | the task must see data state and application logic | with multiple instances, locking is needed, otherwise each will start its own copy |
| Job queue with delayed execution | heavy operations, retries on error, multiple workers | a broker and another component appear, debugging and tracing are more complex |
| External scheduler or build schedule | a single configuration point for a fleet of machines, execution in an isolated environment | network access from an external perimeter is needed, secrets and keys are stored separately |
| Manual execution according to a procedure | rare migrations, one-off reconciliations, supervised operations | a human in the critical path, no reproducibility, the step is easy to miss |
When moving between modes, it is most often the environment that breaks, not the schedule itself.
A crontab line is read almost everywhere, but shell variables, PATH, and user permissions are not inherited: a script that runs successfully by hand fails in the scheduler on a missing binary.
Therefore, next to the task we fix absolute paths, an explicit working directory, and the user, and set the time zone separately — otherwise the clock change twice a year produces a missed run or a duplicate.
Audit and cron setup: how the work is structured
cron tolerates disorder for years: jobs are copied when servers are migrated, duplicate each other, fail silently — output goes nowhere, and the failure is discovered from missing data.
We break down the order of work in which every change is reversible, and the schedule can be read in a couple of minutes.
- Inventory of schedules. We collect jobs from system and user files — including those defined in deployment configuration and therefore not visible in a regular list.
- Schedule audit. We build a timeline of the day: we look for simultaneous runs, overlaps of nightly windows, and tasks whose runtime has grown and overlaps neighbors. We mark critical ones, duplicates, and candidates for removal.
- Environment check. A process from cron does not inherit the user profile, so we check environment variables, working directory, permissions, interpreter path, and access to external services. What is not inherited, we specify explicitly in the task.
- Run scheme. We consolidate the result into a table: what runs, at what frequency, under whose name, where it writes the result, and how it reports an error. The table becomes the reference point for edits.
- Task setup. We rewrite the schedule according to the scheme: spread runs out in time, set overlap locking for long tasks, set a runtime limit and an explicit return code.
- Verification. We run tasks manually and on schedule, compare logs and return codes, emulate failure — unavailable service, insufficient permissions, full disk.
- Handover to support. We provide the scheme, task descriptions, notification points, and rollback procedure, and agree on who makes changes to the schedule and how they are recorded.
Configurations, schemes, and procedures: what remains in the project
After the work, the customer is left not only with a working schedule, but also with a set of artifacts by which it can be maintained further without the author.
Below is what we hand over to the project along with the changes: each item closes its own class of questions when the system is transferred to another team or revisited months later.
- Schedule configuration — tasks are described declaratively and stored in a version control system, not only on the production server. Any change undergoes review, and the schedule is restored on a new node with a single command.
- Dependency map — a diagram of which task starts the next, what each process reads and writes, and where startup order is critical. It shows why shifting one window breaks the entire chain.
- Failure procedure — what counts as a failure, who receives the notification, for which errors the task retries automatically, and for which it stops and requires an engineer.
- Task documentation — purpose, owner, period, expected runtime, and actions to take when it fails. A new engineer gets up to speed in a day, not a month of reading someone else's scripts.
- Task access rights — who edits the schedule, who only views run history. Changes come from separate accounts, not a shared privileged one, so the author of each edit is visible in the log.
- Rotation and retention of results — rules for cleaning logs and reports, limit on archive depth. The disk stops filling up unexpectedly, and old data remains available for as long as needed.
- Rollback plan — how to return the previous schedule version and what to do with tasks that are already running at the moment of rollback.
Case: how task scheduling became predictable
What it was like before the changes
A fleet of dozens of servers, schedules were edited manually directly on the machines — through the standard cron and crontab -e.
Some tasks were duplicated on two nodes, some started on top of the previous execution: backup did not have time to complete, and the next run was already fighting for the same files.
The failure was learned about not from an alert, but from a morning question "why is there no report" — detection time was measured in hours. The share of failed runs held at 24%, and the nightly window of related services was idle for dozens of minutes per week.
What was changed
Schedules were moved into configuration under version control: one source of truth, the same set of tasks on all nodes, rollout by managed release instead of manual edits.
Return code checking, a lock file against repeated execution, a timeout, and retry on network failure were added to tasks.
Logs and results were collected in one place, and alerts were tied to a non-zero return code and to a missed run — that is, to the absence of an event, not only to an error. Steps with dependencies were arranged in a chain: the next starts after the previous succeeds.
What the result was
The failure rate dropped from 24% to 2% — the remaining ones are caused by external services, not the schedule itself. Detection time decreased from several hours to minutes: the alert arrives before the user notices the consequences.
Idle time of related processes in the nightly window was eliminated completely — tasks no longer overlap in time or files.
This framework became the baseline server setup rather than a set of manual agreements, and it transfers to new environments without rewriting.
What to do if cron is already configured but tasks fail?
A schedule that once worked differs from a stable one in that no one checks the execution result. cron honestly wakes the task on time, but the return code goes to the administrator's mail, and errors inside the script go into the void.
Therefore we start with incident analysis: we correlate the failure time with run logs, free disk space, and server load.
Most often the issue is not the schedule itself, but the task environment.
A job started by a timer receives a reduced PATH variable, the home directory instead of the project working folder, and a different set of variables — manually the script runs, but on schedule it fails at the first external command.
We check the environment with a target run: the same user, the same permissions, the same working directory.
The second common cause is overlapping runs. If a task runs longer than the interval, each next instance starts on top of the previous one: two processes write to the same file or table, load grows, and some runs end with an error.
This is treated by limiting concurrency through a lock file, a maximum runtime, and reviewing the frequency.
Next is revision of tasks: we consolidate the existing schedule into a table "what, when, how long it runs, how it behaves on failure" and remove duplicates and dead lines.
We move each task to an explicit log with start time, end time, and return code, and for a fleet of servers we describe the schedule in Ansible — the same set of tasks and the same environment on all nodes.
Bottom line: inventory, a reproducible environment, concurrency control, and failure notification instead of silent failure. A task failure becomes visible within one interval, rather than being discovered by missing data.
Answers to frequent questions about task scheduling
Before work starts, customers ask the same set of questions: what happens when tasks overlap, how to account for time zones, where to look if a run did not happen, and how to move to another server without downtime.
Below are short answers from engineers; each item is refined for a specific infrastructure and recorded in the procedure.
What to do if two tasks overlap?
Overlap is a common cause of corrupted exports: a long nightly report overlaps the next run. We close it with a file lock — the second process sees the resource is busy and exits immediately without touching the data. The skip remains in the log with a return code.
How to account for time zones?
The server calculates time in UTC, while the business sets maintenance windows in local time. We specify the zone explicitly at the beginning of the schedule file, otherwise the switch to daylight saving time produces a double run or a skip. Local time lives only in comments.
How to understand that a task did not run?
Silence from the schedule is a bad signal. We direct task output to a file with rotation, write a start marker and return code, and monitoring waits for a signal within the window: if it does not arrive — alert. The failure is visible in the first minutes, not from a complaint.
How to move jobs to another server?
We store the schedule as code next to the project, so migration is a deployment, not manual line entry. We bring tasks up on the new server, shut them down on the old one, and compare logs run by run so as not to lose a window.
What to do with multiple servers?
A schedule on each node produces double runs and duplicates in data. We leave one scheduler node or a distributed lock in shared storage — the choice depends on how critical start delay is.
Let's discuss task scheduling and come back with a work plan
To make the estimate concrete rather than approximate, to start it is enough to describe the current landscape: which periodic tasks run on the servers, by what exactly — system cron or an application script — and what will happen if a task does not run.
The list of tasks with run frequency sets the scope of work, while the time zone and time synchronization on nodes explain half of the discrepancies in schedules.
- list of tasks: command, run frequency, allowed runtime;
- environments and server roles — where production is, where test is, where replicas are;
- overlap requirements: must a task finish before the next run;
- how the result is currently visible: logs, notifications, monitoring panel;
- windows during which tasks must not be touched;
- server access: connection method, accounts, who confirms changes;
- load constraints: peak hours, heavy exports, calls to external systems.
Based on these inputs, we come back with a work estimate and a draft implementation plan: breakdown into stages, rollout order, rollback points.
Stage approval is done with your side — nothing changes on production servers without confirmation, and we open access for a specific stage and close it after its acceptance. Describe the task — we will come back with an estimate and plan.