Setting Up a Local Environment with Tilt/Skaffold for Kubernetes

Why local Kubernetes diverges from production

A developer edits code, starts a service locally, and sees that the logic works. But validating the change end to end — with neighboring services, queues, and settings of the shared cluster — is only possible after deploying to the shared environment.

The validation cycle stretches out: waiting for the image build, running checks, deploying, reviewing logs — all for one line that needs another fix.

Environment divergence accumulates unnoticed. Locally, there is a different set of environment variables and access permissions, external dependencies are replaced with stubs, and manifests have drifted from those applied in the shared cluster.

Because of this, a failure appears only at the top, and the phrase "it doesn't reproduce locally" becomes a stock one: the engineer goes to read logs in the shared environment instead of debugging on their own machine.

The cost of this scheme is measured not in hours per deployment, but in lost iterations. The longer the cycle, the larger the edits: the developer does not dare validate small steps and brings a large batch of changes to the shared cluster at once.

When something breaks, the cause is searched for not in a specific line but in the whole batch — recovery time grows, and confidence in the release falls.

We explain how this is solved: the local environment is assembled from the same manifests that are applied in the shared environment, starts automatically when work begins — for example, via Tilt — and rebuilds only the affected service when a file is saved.

Change validation returns to the developer's machine, and environment divergence stops being the norm.

What a reproducible development environment provides

A reproducible environment means that local Kubernetes starts from the repository with one command on any workstation. The set of manifests, dependency versions, and configuration parameters live in version control next to the code, not in someone else's memory or in chat.

The engineer gets the same environment as the other participants — divergence disappears before the first launch.

The speed of the edit cycle grows through synchronization of changed files into an already running container. Rebuilding the image and restarting the entire environment are needed only when dependencies change; everything else is picked up on the fly.

An edit reaches validation in seconds, not minutes, and feedback arrives while the task context is still in mind.

Unified manifests remove the main source of divergence: the local environment is described by the same files as the test and production environments, differing only in variable values.

The "works on my machine" class of failures stops being a separate category — if the service does not start locally, it will not start further either, and this is visible before changes are sent.

Engineer onboarding becomes predictable: a new person follows one launch scenario and gets a working environment without figuring out the settings of someone else's machine.

We record this scenario in the repository documentation so that the result does not depend on who conducts the project onboarding.

Approaches to running services alongside the cluster

Local Kubernetes for each engineer and a shared environment for the team solve different tasks, so choosing a launch option starts not with a tool but with the environment: how many services are edited simultaneously, whether environment repeatability between machines is needed, and how important it is to match the cluster in API versions and network model.

The difference between the options shows up in three places: how service images are built on each edit, how traffic gets from the local process to cluster dependencies, and what to do about access permission differences. Below are working options and their limitations.

The choice is usually not either/or: local launch for a fast cycle and a shared environment for integration complement each other.

How we implement and verify the environment

Local Kubernetes is set up once and for the long term, so we go from inventory to handover: each stage ends with an artifact that can be checked.

1. Audit of services and dependencies. We compile a list of components and their connections to databases, queues, and external APIs, noting what is already containerized and what requires manifest changes.
2. Environment diagram. We define namespaces, variables, secrets, entry points, and service startup order — the diagram is agreed upon before the first deployment command.
3. Local cluster setup. We set up an environment for the team's tasks, connect Tilt or Skaffold to rebuild only affected services, and configure port forwarding and local dependencies.
4. Integration with the shared cluster. We compare manifests, limits, and resource names with the production environment so that service behavior locally and in the shared environment does not diverge.
5. Reproducibility check. We deploy the environment from scratch on a clean machine with a second engineer: if the setup runs without manual changes, the environment is considered working.
6. Handover procedure. We provide documentation, launch commands, analysis of typical failures, and update rules, and support the team's first independent launches.

Artifacts that remain in the repository

We describe local Kubernetes as code and keep it in the same repository as the services: then the environment is reproduced by any engineer with one command, not by a retelling in chat. After handover, the team receives the following set of artifacts.

• Environment configurations — values separated by layers: common settings, environment differences, personal engineer overrides. Secrets do not go into the repository; stubs are substituted locally.
• Service dependency diagram — a call graph with notes on what is fully deployed in the cluster and what is replaced by a lightweight stub. You can immediately see which node failure breaks the validation scenario.
• Secret access policies — roles and permissions: who reads values, who changes them, how temporary access is granted. Access revocation does not require manual cleanup of files on the workstation.
• Launch documentation — one short file: requirements, start and stop command, where to look at logs, plus a "if it didn't start" section with typical causes.
• Local development scenarios — Tilt and Skaffold setup: rebuild and file synchronization so that a code edit reaches the cluster in seconds, without manual restart.
• Version update procedure — how often base images and the environment itself are reviewed, who is responsible for updates, and how it is verified that the environment starts after an update.

Case study: where we started and what changed

A product team of fourteen engineers validated edits on a shared test environment: changes from different developers landed in one environment and overlapped. A routine check took 3–4 hours, half the time went to waiting for a free environment and warming up dependencies.

Investigating a defect that did not reproduce locally added another day and usually ended with the words "works on my machine."

We moved validation to a local environment on Tilt.

The environment starts with one command, service composition and variables are described declaratively and match the test environment, and changed code is synced into running containers without a rebuild — only the affected service restarts.

What changed in numbers

Change validation time decreased from 3–4 hours to 10–15 minutes, including bringing up the environment from scratch. Parallel validations grew from one to 9–10 simultaneously: each engineer works in their own environment, there is no longer a queue for the shared environment.

The share of "does not reproduce" failures fell from a third to isolated cases — the environment stopped being a variable that cannot be reproduced. Onboarding a new engineer into working mode decreased from three days to half a day.

How local Kubernetes differs from a shared environment

Local Kubernetes is a personal environment: it starts on the developer's machine, lives from start to stop, and has no obligations to colleagues.

Here you can break manifests, change limits, and rebuild images — the cost of a mistake equals restart time, not a discussion with a neighboring team.

The shared environment is structured differently: it is one cluster connected to by several teams at once, and it reproduces what cannot be obtained locally — external integrations, queues, real volumes, network policies, behavior under load.

The boundary of responsibility is drawn by reproducibility: everything that depends only on code and manifests remains in the personal environment; everything that requires shared data and connections goes to the shared environment.

In the shared environment we keep what must be the same for everyone: secrets and access, database schemas, test data, routing, certificates. Duplicating this in every local environment is pointless — data desynchronization costs more than a few minutes to launch the environment.

Load distribution follows frequency of use: a short edit loop, debugging, and unit checks — locally; integration scenarios, migrations, and version compatibility checks — in the shared environment.

Collaboration between teams rests on simple agreements: who gets which namespace, for how long, who owns shared dependencies, and when they release resources.

Bottom line: the local environment is responsible for iteration speed, the shared environment for environment accuracy, and almost all conflicts grow from mixing these roles.

Technical doubts and answers about the environment

A local environment is almost always stress-tested with doubts: whether it will fit what the team already has, what it will take on developer machines, and what to do with it six months after handover. Below are answers to questions that most often come up at this stage.

What about secrets and database access?

Only templates and structure go into the repository. Real values are mounted from a local file outside version control, and anonymized test data is substituted for the developer. Access is granted by roles and revoked together with the account.

What are the requirements for developer machines?

We calculate the load profile in advance: how many services run simultaneously and which of them are heavy. From this we get a memo for the team — what must be installed and which environments can be started partially so that the laptop remains usable.

Do you guarantee environment compatibility?

We fix parity of runtime and dependency versions between the local and shared environment, and describe configuration differences explicitly. Then "works on my machine" stops being an argument: behavior is reproduced in any of the environments.

What about support after handover?

We provide documentation, configuration diagrams, and the procedure for restoring the environment from scratch.

The team undergoes training on its own project, and the support period and incident response rules are fixed in a separate agreement — without promising "we are always available."

Let's discuss your task and come back with a plan

To base the implementation estimate on facts rather than guesses, we start with analyzing the task. It is important for us to understand three things: service composition, current launch method, and environment isolation requirements. This is enough to propose a scheme and calculate effort without "eyeballing it."

What is useful to send at the start:

• composition of services and connections between them — what should run on the developer machine and what remains in the shared environment;
• current launch method: how images are built, in what order dependencies start, at which step it most often breaks;
• isolation requirements: how many engineers work in parallel on one machine and which environments must not be mixed;
• target branches and environments: how many development environments are needed, how changes are promoted further;
• constraints: closed networks, access to the image registry, third-party services that cannot be run locally;
• readiness criterion — by what signs you consider the environment working and how this is checked in day-to-day work.

In response, we send a work plan: stages, what we do at each, what decisions are needed from your side and at what point. Alongside it — an implementation estimate by timeline and scope of work, so it can be defended within the team. Describe the task — we will come back with a plan and estimate.