Neural architecture search: where manual network selection breaks down
The first week of a project often goes to manual architecture enumeration: engineers assemble dozens of block combinations, train them and compare by accuracy.
Neural architecture search replaces this cycle with controlled search — the operation space is defined formally, and an algorithm drives network selection.
The difference is not in care but in volume. Even a modest space of block types, depths and channel widths yields millions of combinations; only a handful are checked manually, and the choice is almost always random relative to the task.
The cost of error shows up late. The leader model on the validation set does not fit latency and memory constraints on the target device, and rework begins after integration — when the project calendar is almost exhausted.
A separate trap is overfitting on a small sample. When a class has hundreds of examples, the top row of the table is taken by an architecture that accidentally matched the split. That is why candidates are evaluated on several splits and not by one metric.
We calculate accuracy, model latency and weight size jointly, filtering out obviously nonviable options before full training. Selected architectures are fine-tuned and checked on the target device — where the model will run, not on the developer machine.
The result of search is not one "best" network, but a set of options for different latency and memory constraints.
What automatic architecture search gives and where this shows
Architecture search is controlled enumeration: the space is defined in advance (layer types, connection variants, block width ranges), and the strategy — reinforcement learning or evolutionary selection — proposes configurations and filters out weak ones by a control metric. A candidate is checked not only on quality, but also on latency and memory headroom.
The selection cycle shrinks, manual work goes away
Manual enumeration is dozens of sequential trainings with layer edits and restarts. Engineering hours go to mechanics, not hypotheses: comparing several dozen configurations manually takes weeks of calendar time.
Automatic search parallelizes candidate training and early-prunes unpromising branches by an intermediate metric, so the same volume of enumeration is closed in days.
The freed time goes to labeling controversial classes, feature selection, error analysis on slices. The output is not one model, but a list of three to five architectures with different trade-offs between quality and inference cost.
The "model does not fit resources" class of failures goes away
A typical failure: quality on the control set is good, but the model does not fit device or server limits — and the whole pipeline has to be redone.
Search with constraints on latency, memory and artifact size discards such variants at the selection stage, not at the implementation stage. Only candidates that pass resource checks reach the final.
Industry practice confirms the effect: the approach popularized by Google has moved from a research tool to a working way to obtain networks for specific constraints.
Robustness on new data is held by procedure, not luck: the final candidate is fine-tuned on all available data and checked on a holdout fixed before search start — otherwise selection unnoticeably tunes to the test.
Architecture search methods: comparison of approaches and limitations
The choice of architecture search method is a question of engineers' time budget and available compute, not a "best" algorithm.
Options range from manual enumeration to full automatic search: each next step reduces manual work but requires infrastructure for parallel experiments and correct candidate selection.
| Variant | When suitable | Limitations |
|---|---|---|
| Manual enumeration by engineer | Narrow task, 2–3 model families, labels available and clear metric | Limited by team experience and time: dozens of configurations checked |
| Random search | Need an independent baseline before complex search | Does not use accumulated results, spends trainings on weak configurations |
| Evolutionary architecture search | Large solution space, resources for hundreds of trainings | Expensive cycle: each iteration trains descendants fully |
| Search with quality predictor model | One training is expensive, need to select from thousands of candidates | Requires accumulating statistics and controlling predictor bias |
| Transfer of a ready architecture with fine-tuning | Similar domain, little data | Transfer pulls extra layers and wrong assumptions about data |
| Simplification of a trained network | Model already works, need to speed up inference or embed in a constrained environment | Possible quality loss, need metric control at each step |
When manual enumeration is enough
If data is limited and the task class is established, it is reasonable to check three or four typical schemes and stop at the best by metric with headroom for failures.
Such an approach takes days, not weeks, and gives a clear reference point: then it is visible whether automatic search pays off.
When automatic search is needed
Automatic search turns on when manual options are exhausted and the gap between required and achieved quality remains.
Then we fix the solution space, selection metric and stopping rule so search does not turn into endless enumeration: the result must be reproducible, and the best found architecture maintainable.
Implementation stages: from problem statement to model handover
Implementation stages are structured so target environment constraints affect the solution from the very beginning, not surface at acceptance.
Therefore architecture search does not happen in a vacuum but inside frames taken from the task and from the device where the model will run.
What happens at the audit stage
Data and labeling audit is not a formality: we look at the share of labeled examples, duplicates, annotator disagreement and leaks where similar objects end up both in training and validation.
In parallel we fix environment constraints: available memory, response time requirements, launch method. The result is a constraints map that defines the search space and cuts nonviable variants.
- Task and target environment constraints analysis: what we consider success and where the model runs.
- Data and labeling audit: rules, duplicates, class imbalance, sample intersections.
- Search space definition: architecture families, blocks, parameter ranges.
- Search run with early pruning: weak variants leave by fast estimates, resources go to promising ones.
- Training finalists and checking on holdout: fix the protocol in advance, no tuning.
- Measurement on target device: compare quality and response time with what the bench showed.
- Handover of artifacts: configurations, reproduction scripts, report with metrics.
How result acceptance works
We compare the finalist with the baseline model on holdout and separately in the target environment; we explain the discrepancy between bench and device rather than hide it.
Acceptance is closed when metrics are reproduced on your side using our scripts and the conditions under which quality will degrade are clear.
Delivery artifacts: search configurations, schemes, regulations
Delivery artifacts are what remains with the customer's team after project handover. We transfer not only model weights but everything needed for its independent maintenance. Completeness is checked against the list.
Documentation and reproducibility
- Search space configuration: allowed operations, layer ranges, constraints on inference latency and memory.
- Weights of final models and reproduction scripts — a run on the same data gives the same architecture and metrics.
- Model card: purpose, applicability boundaries, known weaknesses, input data requirements.
- Experiment log: which variants were checked, with which metrics, why the final one was chosen.
- Report with measurements in the target environment: response time, memory consumption, quality on holdout.
- Scheme of embedding the model into the product: call points, input data format, behavior when the service is unavailable.
Data and model access rights
The retraining regulation describes triggers — metric degradation, data shift, scheduled update — and who approves launch and how a new version is checked before deployment.
Access policies fix roles, separation of rights to datasets and weights, storage periods and the access revocation procedure when an employee changes.
Case: how we fit the model into target device constraints
The task sounded like a constraint: the same network must run on a device with small memory and response time headroom. There was no separate person for architecture selection in the team — variants were enumerated by the same engineer who led the product.
Architecture search solved here not abstract optimality but whether the solution fits the target device.
| Metric | Before | After |
|---|---|---|
| Variant selection | manual enumeration, weeks | search over architecture space, days |
| Configurations per cycle | dozens | hundreds |
| Response and memory headroom | negative, out of bounds | positive, measurement on device passed |
| Reproducibility | repeated run gave a different set | fixed measurement protocol, result repeats |
What was before
The engineer changed blocks, retrained and measured manually: the cycle took weeks, and weak variants were filtered only at the final measurement — after full training.
Measurements were collected in different places, so comparison of variants between iterations remained non-rigorous.
What became after
Selection was moved to a predictor model: response and memory consumption of a variant are estimated before training, and only promising configurations go to the full cycle.
The search space was limited to blocks supported by the target runtime — obviously nonviable branches are cut immediately.
Search and training were conducted on PyTorch, but the final measurement was on the device itself, not on emulation: the discrepancy between bench and device no longer surfaces at the end.
Customer engineer: "Before I could not explain why exactly this variant was chosen. Now there is a measurement protocol and a list of rejected configurations with the rejection reason."
How to narrow search space under strict latency requirements
Short answer: if target latency is known before start, the search space is narrowed in advance, not the network trimmed afterward. Simplification of an already found model is the second echelon, when device constraints became clear later or change between releases.
In practice the first path is cheaper: search does not burn runs on configurations that will not pass inference time anyway.
The search space is narrowed along three axes: depth and width bounds of blocks, a fixed set of operation types and inherited constraints from the hardware profile. Target latency is built directly into the selection criterion, not counted as a penalty after training.
The profile is taken on the real device or a close analogue — for example, via TensorFlow Lite: this cuts architectures that look fast on the bench but do not fit target hardware.
The reverse order is justified when device constraints are not yet fixed.
First quality is searched for, then model simplification is applied — pruning, distillation, reduced computation precision — and metrics are returned via manual fine-tuning: weight selection, additional training of individual blocks.
Change one technique per run and compare on holdout — otherwise quality loss is masked by averaging.
Too narrow a search space cuts off nonstandard solutions — branched paths, nonstandard connections. A reference from our projects: 2–3 operation types, depth spread within the real task range, and anything that obviously does not pass latency — set as a constraint, not a penalty.
Common technical questions: data, hardware, reproducibility
The questions below surface not at the idea stage, but when an engineer starts counting resources: how much data is really needed, what happens to an already trained model and whether search can be conducted inside a closed loop. The answers determine whether the solution remains a prototype or reaches production operation.
How much data is needed for architecture search to have an effect?
Neural Architecture Search does not replace labeling — it searches for network structure for the task. On small samples architecture enumeration degenerates into fitting, so we start from a volume where the metric is already stable and add an unlabeled corpus for pretraining.
What will happen to an already trained model?
It remains the reference point: the search space includes the current architecture, and a new version is accepted only with a stable gain on holdout. If there is no gain — we return to the previous model, and investments in data and labeling are preserved.
How do you exclude fitting to the test set?
The test set does not participate in search or parameter selection — it opens once for the final measurement. Selection happens on validation, plus we check metric stability on several splits: without this the result does not reproduce on new data.
Can search be conducted in a closed loop?
Yes. Search and training are deployed on your infrastructure, without accessing external services: data and weights do not leave the loop. Only metrics and search configurations go outside — if necessary, via offline media.
Let's discuss the task: what to send for assessment and plan
To return with an assessment and work plan, not clarifying questions, we need a slice of the task.
In neural network architecture search, the outcome depends not on compute power but on how precisely the input and boundaries are defined: the search space is built from them, operations and cost function are chosen. The more specific the input data for calculation, the fewer iterations will go to alignment.
- Problem statement: what we feed as input, what is considered the correct answer, how the customer measures quality on their side.
- Model requirements: target accuracy, allowable response latency, memory footprint, maximum final model file size.
- Target device characteristics: server, mobile device or embedded controller with a hard memory limit — the search space itself depends on this.
- Data examples: depersonalized sample slice, labeling, class distribution, separately — rare and borderline cases where the model errs most often.
- Current model or baseline version: its metrics, error locations, constraints that cannot be lost during rework.
- Runtime environment constraints: export format, available accelerators, requirements for offline operation without network, data processing regulations.
Describe the task in free form and attach what you have — we will clarify the rest. Based on these inputs we will return with an assessment and work plan: architecture search stages, acceptance criteria and the point where the customer joins. The next step is agreeing on quality criteria, then work proceeds according to the fixed plan.
