NeRF development: where classical 3D reconstruction breaks
Classical multi-view photogrammetry relies on matching feature points between frames. As long as the surface is matte, rigid, and textured, a pipeline like Agisoft Metashape converges predictably: the point cloud is dense, the mesh is built without manual editing. The limitations of photogrammetry begin where there is nothing to match or nothing to trust.
Shooting complex scenes breaks this pipeline systematically.
Gloss, glass, polished floor, and chrome produce highlights that the algorithm mistakes for its own texture and smears across the surface; uniform walls and white plastic provide not a single stable point; wires, grids, and foliage are eaten away during decimation. Indoors and in low light, sensor noise is added, and some points fail the confidence threshold.
The client sees it immediately: holes in the geometry, “lumps” from discarded points, drifting seams at joints, and blurry textures where frames averaged out. On a product card, the result is a rotating model with a ragged edge; on a building flyaround, a collapsed corner.
Reshooting does not guarantee a fix: if the scene is out of tolerance, a second pass will produce the same artifacts.
Neural 3D reconstruction shifts the task from point matching to training a density and color field that accounts for view-dependent appearance. This is where robustness to glare and thin structures comes from, along with continuous viewpoints between frames.
But the burden shifts to data discipline: coverage density, stable camera poses, consistent exposure. We’ll break down how this is handled in NeRF development.
What neural reconstruction delivers: measurable effect
Neural reconstruction seems slow and “raw” only until the first proven process.
Once a scene is trained once and the result is delivered as a ready representation, the cycle reaches the level of a shooting day: several hundred frames turn into a coherent model in hours, not days of manual assembly.
The NeRF approach delivers the main thing — reproducibility: the next scene follows the same route without rebuilding the process from scratch.
Five effects by which implementation is visible in numbers:
- Reconstruction speed. From a set of images to a model — hours; geometry remarks are resolved the same day, not by next week.
- Geometry accuracy. Deviation from control measurements is fractions of a percent of the scene’s overall dimensions, so the model is suitable for measurements, not just for a picture.
- Share of usable viewpoints. 85–95% of frames make it into the model versus about half when shooting “by eye”: dropouts at frame edges and occlusions no longer reject an entire shot.
- Elimination of artifacts. Entire classes of defects disappear: dropouts on glossy and uniform surfaces, noise along the edges of thin objects, texture jitter between frames.
- Synthesis of new viewpoints and less crew downtime. The camera trajectory is set after shooting, so an additional shooting day is replaced by configuration, and the share of scenes requiring a return visit drops from every third to isolated cases.
Overall, this changes the place of reconstruction in the project: it stops being a separate long stage and becomes a regular link in the production chain, where overhead costs for trips and reshoots stop growing with each new scene.
3D reconstruction formats: NeRF development and alternatives
Reconstruction methods differ not in “quality” but in what they assemble the scene from and what they output. One reconstructs geometry from a set of overlapping images, another relies on a dense point cloud, and a third combines both sources. Below are the applicability boundaries of each option without marketing rounding.
| Option | When it fits | Limitations |
|---|---|---|
| Neural reconstruction (NeRF) | Shooting with a regular camera or drone, photorealism and free flyaround of the scene are needed | Requires dense viewpoint overlap, training takes hours, scale is set by external references |
| Gaussian primitives | Interactive delivery is needed: the scene is displayed in a browser without pre-rendering | Heavier in memory, sharp boundaries blur, collisions and scene edits are separate work |
| Laser scanning | Measurements, documentation, deviation control — dimensional accuracy matters | Requires a site visit and equipment, little data on color and material, mirror surfaces produce noise |
| Hybrid reconstruction | A single project needs both precise dimensions and a photorealistic look | Harder to align coordinate systems and calibrate the combined setup, longer preparation |
The choice is usually dictated by the data source. If shooting has already been done with a camera, neural models and Gaussian primitives are compared: the former delivers correct viewpoints, the latter smooth rendering.
When there are accuracy requirements, laser scanning is indispensable, while a hybrid covers both tasks at once — measurements and a presentation view in one scene.
Work stages: from scene audit to model handover
The risk of error in a 3D project grows nonlinearly: insufficient viewpoint overlap or glare on glossy surfaces is discovered only after lengthy training, and the scene has to be shot again.
That is why the stages of a 3D project are arranged as a chain with a verifiable result at each step.
- Object audit. We inspect the scene: dimensions, share of reflective and transparent surfaces, lighting character, access to shooting points. We record areas that definitely need additional viewpoints and diffused light.
- Camera placement plan. We calculate the number and geometry of points, overlap of adjacent frames, and reference markers for calibration. The plan is agreed before the site visit — this removes the risk of reshooting due to “holes” in coverage.
- Test shoot. We shoot a fragment — the material-wise most complex area — and run the full processing cycle. Defects are revealed before the entire site is shot.
- Solution assembly. Data preparation, calibration, NeRF model training, and optimization for the target platform: web, mobile app, or offline viewing.
- Quality control. We compare synthesized viewpoints with control frames that did not participate in training; we separately check geometry and “floating” artifacts at scene boundaries.
- Model acceptance. We run the agreed list of criteria on real usage scenarios: problematic areas are further trained or closed with targeted reshooting.
- Handover. We deliver the assembled model, camera placement plan, reproducible configurations, and reshooting instructions — the scene can be updated without us.
Handover package: configurations, plans, regulations
Handover does not end with a demonstration of the finished scene. The client receives a set of artifacts that the team can use without us to deploy the environment, shoot a new object, and repeat NeRF model training with comparable quality. Below is what exactly the package includes.
- Environment configurations: build manifests, pinned library versions, training launch parameters — deployment on your own infrastructure without contacting us for clarifications.
- Shooting regulations: scene traversal trajectory, overlap between frames, tolerances for exposure and focus stability. This is material that cannot be reconstructed after the fact — reshooting the scene later will no longer be possible.
- Lighting plan: placement of sources on site, acceptable light types, measurements on a reference scene, and signs at which shooting is stopped and lighting is rearranged.
- Access rules: roles with the right to launch training and roles with view-only access to results, an operations log, and the procedure for issuing and revoking keys.
- Project documentation: data structure, result export formats (mesh, point cloud, Gaussian primitives), and model applicability boundaries by scene class.
- Support regulations: how to update dependencies, what to do when metrics diverge after training, and where rollback points to the previous model version are located.
- Reproducibility of the assembly pipeline: pinned versions, an environment container, and an acceptance checklist — a repeated run on the same data yields a result within the stated tolerance.
Each block is verified at handover: we run the full cycle “new scene — new result” strictly according to your regulations. If the documentation diverges from reality, we fix either the configuration or the text — before final acceptance.
Case study: complex scene, metrics before and after
A production unit with mirror housings, glass inserts, and thin reinforcement twice resisted classical shooting. Photogrammetry loses reference points on shiny surfaces: the model was assembled in fragments, glare left holes, and thin posts disappeared between viewpoints.
Starting point
Only 38% of the captured viewpoints made it into the reconstruction, and surface coverage did not exceed 68%.
Bottlenecks had to be reshot three times, and about six weeks passed between the site visit and the first usable model — half of that time went to manual stitching and filling gaps.
What was changed
Shooting was switched to calculated trajectories with specified frame overlap.
Instead of searching for correspondences between images, a neural scene model (NeRF) was trained: it reconstructs density and color at every point in the volume, so glare and translucent inserts stopped being gaps.
Training ran on NVIDIA accelerators, with frame parsing and edits in a single pipeline, without manual stitching.
What was achieved at the output:
- share of usable viewpoints — from 38% to 91%;
- reshoots — from three to zero: disputed areas are computed from existing frames;
- surface coverage — from 68% to 97%, including reinforcement and glazing;
- time from shooting to finished model — six days instead of six weeks.
The metrics were taken on the same object and the same set of viewpoints — the only difference is the reconstruction method. If an area is unsatisfactory, the model is retrained from the existing source data, without a new site visit.
What to do if the scene is already shot and there is a point cloud?
Short answer: yes, a shot archive and point cloud are almost always reused — but as source data and a geometric reference, not as a finished result.
Photogrammetry has already provided camera poses, scale, and sparse geometry; NeRF adds a dense scene description and the ability to build viewpoints that simply did not exist in the old shoot.
What transfers without loss: camera calibration, coordinate system alignment, object masks, scene partitioning into blocks.
What is reassembled: the cloud is converted into depth maps and works as a hint during training — the model relies on real geometry and does not “float” on uniform walls and glare, where pure photogrammetry produced ragged holes.
What is usually missing is overlap between frames and viewpoints from above or below. This is supplemented by targeted additional shooting, usually 10–20% of the original volume, without re-traversing the entire scene.
Moving objects from the archive are cleaned out with masks; otherwise they smear across the volume and spoil the geometry.
Model fine-tuning is based on weights already obtained on similar scenes: basic patterns transfer, and details are refined on the specific object — training converges faster than from scratch and requires fewer new frames.
The output is a photorealistic model that renders new viewpoints and remains compatible with previous measurement data: switching from photogrammetry does not reset what has already been done.
Technical limitations: transparency, glare, mirrors
The method relies on pixel consistency between viewpoints: the same scene point must look the same from different cameras — then the model confidently places it in the volume.
Where physics breaks this connection, the network does not stop but fills in something plausible: hence the “blurry” glass and jittery reflections in the finished model.
The limitations of neural reconstruction almost always come down to transparency and mirrors. Glass and water do not return a stable texture, and reflection depends on camera position, so the model averages it into a blurry patch.
Workarounds: a polarizing filter, temporary matting of reflective areas, a contrasting background behind a transparent object, and additional viewpoints at angles where the reflection is unambiguous.
Moving objects in frame break the basic assumption of a static scene: a pedestrian or swaying foliage turns into a “ghost” of several positions.
In NeRF development, this is handled by camera synchronization, dynamic masking, and a separate pass over the object — the background and the moving part are reconstructed independently.
What definitely will not be reconstructed?
Uniform and thin structures: wires, mesh, wire fences, solid-color walls. And anything that did not appear in any frame — there the model fills in a guess, not a fact.
Can frames be shot later?
Yes, if camera calibration and lighting conditions are preserved. Under different light, new frames conflict with the already assembled volume and produce noticeable seams.
Let’s discuss the task: what is needed to evaluate your scene
For the reconstruction project estimate to be substantive, we need inputs on the scene, shooting, and the point where the result will be integrated. Below is the minimum set sufficient for us to come back with an implementation plan, scope of work, and acceptance criteria.
- What we are reconstructing: type of object, its scale, how many scenes are in the project, static shooting or an object in motion.
- Source materials: frame archive, point cloud, results of previous measurements, requirements for export formats.
