Pods Stuck in Pending: Where Kubernetes StorageClass Breaks
The deployment looks healthy: manifests applied, service created, but pods are stuck in Pending and not moving. In pod events, there is FailedScheduling referencing an unsatisfied volume claim, while the controller reports a volume provisioning error.
The application does not crash: it simply does not start because Kubernetes is not giving it storage.
The cause is almost always in the storage class configuration. If the StorageClass is set to immediate binding mode, the volume is created before the scheduler selects a node — and it easily ends up in a zone where the required disk is unavailable or incompatible with that node.
With delayed binding, the volume is provisioned for a specific node, and a stuck deployment often means there is nowhere to place the pod, not that storage is unavailable.
A separate branch is a mismatch between claim and class: the PVC references a name that does not exist in the cluster, a class without the default class label, or requests an access mode that the driver cannot provide.
Then the volume is not created at all, and the autoscaler cannot add a single replica: each new one waits for its volume, and scaling stalls along with the rollout.
Diagnostics here are predictable: we look at PVC and pod events, compare the class name, binding mode, and reclaim policy, and check that the driver is registered and there is free capacity in the selected zone.
The next task is not to wake up a single stuck pod, but to bring storage policies into order — we will go through how this is resolved.
What Dynamic Volume Provisioning in a Cluster Provides
Dynamic volume provisioning is built on one linkage: the StorageClass describes the storage class and its reclaim policy, while the PersistentVolumeClaim in the application manifest simply references that class.
Then the storage controller itself creates the volume, binds it to the claim, and attaches it to the pod.
The manual step “order a disk, wait for allocation, enter the ID into the specification, and redeploy” disappears from the process entirely — the volume appears in seconds, not hours of approvals.
For deployment time, this changes more than it seems. The volume claim travels in the same commit as the application itself, so a new environment in the test loop comes up with a single manifest apply.
The developer does not wait for an infrastructure request, and the test environment does not live in the state of “the pod is already there, but the disk is not yet.”
Scaling the workload gets the same effect: as the number of StatefulSet replicas grows, each new pod automatically creates a volume from the class and mounts it, without requiring manual work for each copy.
The class of failures such as “volume not provisioned,” “claim hanging in wait,” “disk bound to the wrong zone” goes out of operation because the controller closes them, not an on-call engineer.
Existing static volumes continue to work: classes and claims for them coexist, so the transition can be done gradually, one application at a time.
If StorageClasses with data retention policies are already described in the cluster, new claims use them as is — compatibility with old manifests is preserved, and there is no need to change the storage layout.
The share of manual operations drops to isolated cases: only those decisions where a human is needed remain manual — quota size, lifetime, and the policy for deleting a volume after the claim is removed.
Everything else — creation, binding, mounting — happens automatically, and the blast radius narrows to a single claim instead of a chain of manual steps.
Comparison of Volume Provisioning and Binding Modes
The choice between static and dynamic provisioning is a choice between predictability and speed.
Volume provisioning modes determine who creates the medium: the administrator describes volumes in advance and keeps them in a pool, or the controller creates the medium when a request from a pod appears.
Static provisioning gives full control over where data lands; dynamic provisioning removes manual work for every new service.
Volume binding is influenced not only by the mode, but also by the volume reclaim policy: after the claim is deleted, the medium is either returned to the pool, retained as is, or deleted along with the volume. This affects both rollback capability and how quickly storage volume grows.
When Each Mode Is Appropriate
| Option | When It Fits | Limitations |
|---|---|---|
| Static provisioning | Data outlives the cluster: databases, archives, volumes with an external lifecycle | Volumes are created manually, the pool must be kept with reserve, name-based binding is fragile |
| Dynamic via the default class | Homogeneous workload with no requirements for the medium profile | All requests get one profile; changing it affects the whole cluster, not just one team |
| Named classes for profiles | Different requirements: fast disks for transactions, capacious ones for logs and analytics | More objects to maintain, each class has its own reclaim policy |
Delayed binding deserves special mention: the volume is created only when the pod is already placed on a node. Otherwise, in a multi-zone cluster, the medium may end up in the wrong zone, and the pod will hang waiting, while diagnosing such a state takes hours. We will go through how to choose a mode for the workload profile and not lose data during rollback.
How We Configure Kubernetes StorageClass: Work Stages
We structure the work order so that storage classes come from real workload profiles, not guesswork. First, we record what is already mounted and how volumes behave under load, then we design the class scheme, and only after that do we change the cluster. Each stage is closed with an artifact that can be verified — before moving to the next.
- Kubernetes volume audit: we collect PVCs and PVs by namespace, review the driver, reclaim policy, binding mode, actual utilization, and node topology. The output is a workload table with requirements for latency, shared access, and I/O intensity.
- Designing the class scheme: we identify 3–5 profiles — fast volumes for databases, capacious ones for logs, archive ones with a cold tier — and assign each its own StorageClass with explicit parameters.
- StorageClass configuration: we set the reclaim policy, expansion permission, delayed binding for volumes in multiple zones, and size limits so that a single PVC cannot drain the entire pool.
- Preparing and migrating workloads: we assign the default class and move existing volumes by changing the class through recreation or snapshots — without long service downtime.
- Testing on the bench: we run through node failure, quota exhaustion, volume expansion on a live workload, and rollback to the previous class. We record how the scheduler behaves in each scenario.
- Production storage implementation and handover to the team: we roll out by namespace, hand over the class scheme, volume request regulations, and instructions for the on-call shift.
Access Policies, Quotas, and Objects in the Cluster
After implementation, the customer’s team is left not only with working storage classes, but also with artifacts by which the cluster is operated without us. Below is what is handed over and checked at acceptance.
- Descriptions of each StorageClass: purpose, parameters (volume binding mode, data reclaim policy when a claim is deleted, disk type), and the workload class it was created for — a developer selects a class by description, not at random.
- Access policies: role-based separation of permissions — who creates and edits classes, who only requests volumes, who reads status; separate roles for service accounts.
- Namespace quotas: ResourceQuota and LimitRange for each namespace — a limit on total storage volume, number of claims, and size of one volume. When the quota is exhausted, the team gets a clear error, not a stuck task.
- Volume lifecycle rules: which classes retain data after claim deletion, which release it; how to take a volume for incident investigation.
- Maintenance regulations: procedure for adding a class or editing parameters, bench testing, change application window, rollback points.
- Cluster documentation: placement scheme, list of classes, manifests and their history in version control, change log.
- Monitoring and alerts: which metrics on volume claims and free space are collected, at which thresholds a notification is sent.
- Team handover: walkthrough with the customer’s engineers, run-through of typical scenarios, escalation contacts during support.
Case Study: Eliminated Volume Hangs Under Peak Load
Before the project, volumes in the cluster were provisioned manually: on request, an engineer created a disk and inserted it into the manifest, while pods waited their turn.
During peak sales, the queue grew, some pods remained in Pending until morning, and the catalog section was unavailable.
The Kubernetes scheduler could not help — without a StorageClass it had nothing to bind the volume with, so the on-call engineer handled such incidents at night, manually.
What We Changed
We described StorageClasses for workload profiles: a fast default class for databases and queues, a cold one for logs and exports.
We enabled dynamic provisioning, volume binding to the pod’s zone, allowVolumeExpansion for growing a volume without recreation, and reclaimPolicy Retain for critical data — the volume no longer goes away with the claim.
On top, we set quotas on request size so that a single pod cannot consume the entire pool, and volume snapshots before release for fast rollback.
Metrics Before and After
| Metric | Before | After |
|---|---|---|
| Volumes in Pending at peak | up to 18% of requests | zero |
| Volume provisioning | 4-6 minutes, at peak — dozens | seconds |
| Manual on-call operations | 10-12 per week | 0 |
| Storage-related incidents | 3-4 per month | 0 in six months |
An entire class of failures is gone: the workload no longer queues for a volume but triggers automatic scaling. The on-call engineer spends night hours on planned work, not manual disk binding, and releases no longer slip because of storage.
What to Do If a Volume Is Already Created Statically
The storageClassName field of an existing PVC is immutable — you cannot edit it on a live cluster.
Therefore, the transition to dynamic provisioning is done not by editing the manifest, but by migration: a StorageClass with the required driver and reclaim policy is created, a new PVC is prepared, data is transferred, and only then is the workload switched to the new volume.
The transfer method depends on the storage driver’s capabilities. The careful path is a volume snapshot or cloning with restore into a new PVC: the copy is taken at the storage level, the application continues to run on the original volume and does not know about the copying.
If there are no snapshots, a second replica is brought up that populates the new volume while the service responds to traffic.
Reusing the old volume as is will not work: a static PV is bound to a specific PVC.
A StatefulSet will not pick up someone else’s PVC automatically — volumeClaimTemplates is edited, and pods are replaced one by one with verification that the replica has caught up and the data on the new volume is consistent.
The old volume remains mounted for rollback until the new scheme has completed a full cycle under load.
On a live cluster, you cannot change accessModes and class binding of a PVC, and size can only be increased, and only if expansion is enabled in the StorageClass; some drivers do this on the fly, while others require a pod restart.
The resource reclaim policy is set in the StorageClass before volume creation: editing it on a running PV is risky — deleting the PVC will take away data that the application still references.
Technical Doubts Before Starting Work
Doubts are usually not about Kubernetes as such, but about how a specific class will fit the existing storage, volumes, and team permissions. Below are the questions engineering teams ask most often before starting.
Will the class work with our storage?
A StorageClass is a profile, while the actual disk work is performed by the storage driver. During the review, we look at which access modes it supports and how it behaves when a node changes, and we bring up a test class on an isolated loop — before any changes in production.
What will happen to existing volumes?
Created volume claims remain bound to their class — the new class does not rewrite them. We migrate through a data copy and a new claim, not by editing a live volume.
We pay special attention to the default class: while the label is set, all volumes without an explicit specification go to it.
Why can’t we get by with one class for everything?
The limitations are set by the driver: one volume — one node, shared access, behavior when a deleted claim is removed. That is why classes are split by profile. A universal class either does not provide the required mode or deletes data where it needed to be retained.
What access rights do teams need?
A class is a cluster-level resource; it is created and edited by the platform team, not by every developer. We grant application teams rights to volume claims within their namespace and quotas on volume: classes remain manageable, and access to storage does not sprawl.
Let’s Discuss the Task: What to Prepare for the Conversation
To discuss the task substantively, it helps to have the current storage picture and requirements for it on hand.
We are not asking you to formalize a full technical specification — the list below is enough, and from it we can already assemble the work estimate and implementation plan.
- Current class scheme: which StorageClasses are declared in the cluster, which one is marked as default, which volume reclaim policies are set, and where the physical disk goes after a claim is deleted.
- Workload profiles: which services write sequentially (logs, archives), which need random writes (databases), and which are fine with a network disk — this determines the provisioning type and the volume-to-node binding mode.
- Access requirements: whether simultaneous writing from multiple nodes is needed, whether there is a hard binding to an availability zone, and what rights teams and namespaces have.
- Growth and copy handling: whether on-the-fly volume expansion is allowed, whether snapshots are taken, and how backups and restore from them are arranged.
- Procedure for verifying the result: on which test loop we compare behavior, by which scenarios — volume creation, workload migration, disk release after deletion, node failure.
- Operating mode: maintenance windows, fault-tolerance requirements, who on the customer side accepts changes and signs off on the transition to the new scheme.
If some data is missing, we reconstruct it ourselves during the survey stage — this is reflected in the implementation plan as a separate line, rather than being pushed onto your team.
Send what you have: even with an incomplete set, we will still come back with an estimate and a clear work order.