Improving AI audio quality: where recording breaks
Defects rarely come one at a time: a recorder in a meeting room adds ventilation hum, a laptop microphone adds overload, a call adds compression artifacts.
Therefore, improving AI audio quality begins not with choosing a model but with an inventory of the recording chain: what exactly is damaged, at what stage, and whether it can be restored at all.
Noise in speech recordings can be stationary — ventilation hum, power-line interference — or impulsive: table knocks, keyboard, rustling clothing.
Stationary noise is subtracted based on an estimate of the noise spectrum and disappears almost without a trace; impulsive noise requires detection and interpolation.
Rough filtering leaves a "musical" overtone and eats away consonants, on which the intelligibility of the recording depends.
Clipping and overload are a fundamentally different class: the waveform is cut off at input, and the source data is lost.
The model reconstructs a plausible envelope from context but does not restore the signal, so for such fragments it is more honest to state the limitation than to promise a complete result.
Loss of highs and compression artifacts behave the same way: lossy codecs discard quiet details and the upper range — at the output you hear a metallic overtone and a "floating" band.
Upscaling reconstructs the band probabilistically, and the ceiling of quality is set by the source, not the model.
The practical criterion is the intelligibility of the recording, not "it sounds cleaner." It is measured by the proportion of correctly recognized words and the number of edits in the finished transcript; for subtitles, meeting minutes, and call analysis the thresholds differ.
In broadcasting, a similar task is solved reproducibly: industry recommendations EBU describe loudness normalization and intelligibility control. A specific processing is selected for each defect — this determines what it even makes sense to promise for a specific recording.
What processing delivers: measurable effect on an archive
We accept processing not by ear but by measurements on the same material: a fragment is run before and after, measurements are repeated across the entire length of the recording, not on a convenient excerpt.
Speech intelligibility increases due to suppression of stationary noise — ventilation hum, power interference, equipment background.
We separately check whether noise reduction "eats" consonants and sibilants: overly aggressive filtering produces silence instead of speech, so the threshold is selected by a metric, not by eye.
For the set of operations, we use manual remastering in iZotope RX as a reference — the difference is that ours runs in batch across the entire archive.
The share of manual editing is the main practical indicator. On noisy tracks, an editor used to go through every file; now only what failed the noise and overload threshold goes to rework.
The share of such recordings is measured as a percentage of the array and drops severalfold; along with it, the number of repeat recording sessions decreases.
Loudness stability is provided by normalization to the broadcast standard: the spread between takes and releases is reduced, and the listener does not have to adjust the control.
The overall result for the archive is suitability: the share of materials that can be returned to circulation or published without re-recording grows, and classes of failures such as overload and level dropouts are eliminated at the primary processing stage.
Processing modes: batch, streaming, studio
Batch and streaming processing solve different tasks, and confusing them is the most common reason an AI audio quality improvement project stalls halfway.
In batch mode, the file is available in full: the algorithm sees the track from beginning to end, so denoise and remastering are built on the full context of the recording.
In streaming mode, it works on a short window and must fit within the latency — quality here is limited not by the model but by how much data it is allowed to see.
| Option | When it fits | Limitations |
|---|---|---|
| Batch across an archive | Digitizations, podcasts, collections — where the end result matters, not the moment of delivery | There is a pause between the run and delivery; the entire array is not processed instantly |
| Streaming | Recording and live broadcast, where the result is needed immediately | Short context window, repeat run is impossible, an error goes into the signal |
| Selective remastering | Individual defective tracks within a large array | Requires selection of positions and listening control for each |
How to choose a mode
The difference is visible on defects: broadband noise in batch is removed predictably, while in streaming part of it turns into a musical artifact that can no longer be removed.
The mode is selected by the type of material, not by volume: for a mixed archive, the working scenario is a batch run across the entire array plus selective remastering where automatic processing did not pass listening acceptance.
We fix the processing boundaries before the start so that disputed tracks do not surface at the delivery stage.
Work stages: from audio recording audit to delivery
Audio recording audit is the first step: from a sample you can see where noise lies outside the speech range and where it masks the useful signal, and any denoise there will remove the timbre along with the artifacts.
Then we assemble a processing scheme for the specific material, not for a general preset: mains hum, tape hiss, broadband hall noise, and clipping from overload require different operation orders.
Below is the order of work; the scope of the audit and the number of control fragments depend on the archive duration.
- Audio recording audit. We mark up the sample by defect types, examine spectrum and dynamics, and record fragments where restoration hits the physical limit of the source.
- Processing scheme. We determine the order of denoise, declick, remastering, and upper-frequency upscaling, the control points between them, and the modes for preserving the original balance.
- Profile setup. We select suppression depth and thresholds separately for speech, music, and mixed tracks so that quiet parts do not sink into noise.
- Check on control fragments. We compare before/after on lines, pauses, and transitions — looking for "musicality" of artifacts, timbre changes, and overload at peaks.
- Control listening. We run the result on different playback systems, from headphones to club acoustics, and adjust the profile if the defect is audible only on one path.
- Delivery of the result. We provide master versions in agreed formats, fragments for comparison, and a report with a processing map — it allows edits to be made without a repeat audit.
What you keep: models, setting profiles, reports
The processed archive is only part of the result. Along with it, you receive everything needed to repeat the processing on new recordings without our involvement: models, settings, reports, and instructions for your team.
- Model sets — configurations selected for your recording types: indoor speech, outdoor interviews, digitized tapes. The set is fixed, so the result is reproducible on new material.
- Setting profiles — noise reduction, loudness alignment, and intelligibility restoration parameters combined into ready-to-use profiles. The operator selects a profile by source type instead of manually selecting values.
- Quality reports — for each file you can see what changed, which defects remained, and where manual editing was required. They are used to accept the work or return it for revision.
- Processing procedure — the order of steps from source intake to archive delivery, with control points and file naming rules.
- Team instructions — how to launch processing, what to do with problematic recordings, and where to look if the result deviates.
- Version and rollback point log — which processing version was applied to each batch; you can compare two variants and return to the previous one.
- Storage and access scheme — where sources, intermediate, and final files are stored, who has access, and how to transfer the archive to another team.
Case: restoring an archive of interviews and lectures
Initial state
The archive was about a thousand hours of lectures and interviews recorded at different times and with different equipment.
Speech was masked by equipment hum, carrier hiss, and level overload on loud fragments, so transcription was done by ear and final editing manually, frame by frame by sound.
Only about three out of ten recordings were suitable for publication without editing, and an editor spent up to four hours cleaning one hour of interview.
What was changed
Processing was divided into two independent passes: first, restoration of the signal shape — click removal, level alignment, overload suppression — then adaptive noise reduction using a noise profile taken from pauses between phrases.
Speech intelligibility was raised separately from musical and noise layers so as not to "eat" the voice timbre; final files were brought to a single broadcast format.
We worked in iZotope RX together with our own batch processing scripts; manual finishing remained only for fragments with overlapping voices.
What became
The share of recordings suitable for publication without editing grew from 30% to 88%, and now one hour of material takes 20–30 minutes of manual work instead of several hours.
Repeat exports decreased: editors stopped returning to the same file two or three times because master versions were immediately saved with separate tracks.
The archive became searchable by transcript — old interviews went into reuse: training courses, podcasts, quotes for publications.
How denoise differs from remastering and upscaling
These three tasks are placed in one row because their input is common — a recording with defects. But their goals differ: denoise cleans, upscaling reconstructs, remastering reassembles. The confusion is costly: one step is expected to deliver the result of another.
| Task | What it solves | Applicability boundaries |
|---|---|---|
| Denoise | Suppresses stationary noise: hum, hiss, interference, carrier noise | Does not return what is lost: with deep suppression, an "underwater" timbre and musical artifacts appear |
| Upscaling | Synthesizes upper spectral components that are not present in the recording | Reconstruction, not the original: on a dense mix, synthesis produces an overtone inseparable from the signal |
| Remastering | Reassembles balance, dynamics, and intelligibility | Does not cure overload and clipping, and excessive compression eats dynamics |
Applicability boundaries are more important than declared capabilities. Denoise works with what was recorded: if the useful signal is drowned in noise, separation will pull it along too.
Spectrum restoration is probabilistic reconstruction, and its quality depends on the material: speech and solo instruments are restored more reliably than a dense mix.
The order of steps is part of the method: first denoise, then spectrum restoration, then remastering. The reverse order will fix noise reduction artifacts, and separating them from the signal will no longer be possible.
Before choosing a chain, we listen to the material and examine the noise profile and band. Often one step is enough: on a clean recording, extra processing is more audible than the original defect.
Common technical doubts before starting work
The objections here are usually engineering ones: what can actually be pulled out of a compressed file, whether a metallic tint or "wateriness" will appear, where the sources go.
We answer each point before the start — the processing scheme and requirements for materials depend on it.
The input is rarely a neat master. More often it is a mix of dictaphone recordings, broadcast tapes, digitized cassettes, and tracks from editing.
Therefore, we first build a material map: where frequency bands are lost, what we restore with a model, and what we clean with classical methods.
Can anything be pulled out of a compressed source?
Yes, but not everything. The codec cut off the upper bands — restored frequencies are synthesized by the model from harmonics rather than taken from data.
In the requirements for materials, we record what can actually be returned: speech intelligibility, air in cymbals, depth of scene. What was destroyed at mastering does not come back, and we say this immediately.
Will side artifacts appear?
They appear if you drive noise reduction with one knob. We keep separate paths — noise reduction, band remastering, loudness alignment — and listen not only to the cleaned track but also to the difference signal: you can hear what exactly was removed.
Cut consonants and "underwater" overtones are caught at acceptance using control fragments.
Do you work with confidential data?
Yes. We deploy models in a dedicated isolated environment, with role-based access; copies do not leave your perimeter. Upon completion, we provide a processing log showing who opened materials and when, and delete working copies.
Let's discuss the task: what materials to send for assessment
To come back with a scope assessment and work plan, rather than general wording, we need input material.
From sample recordings, it is immediately clear where the boundary lies between denoise, remastering, and restoration of lost frequencies — and which steps even make sense for your material and which would be unnecessary processing.
- Recording samples — 3–5 of the most indicative fragments of 30–60 seconds each: both quiet sections and loud ones, and those where the defect is most audible.
- What exactly interferes — hall noise, mains hum, tape hiss, clipping, a "telephone" timbre: our engineers compare your description with the spectrum and clarify the task.
- Recording conditions — what was used to record, in what room, whether there was extraneous background; this determines what can be cleaned without losing the voice.
- Target medium — podcast, video, archive, calls: the permissible degree of processing and intelligibility requirements depend on it.
- Result reference — an example of "how it should sound" and a list of what must not be lost: the speaker's timbre, breathing, intonations.
- Scope of material — how many hours or files, how they are stored, and whether copies can be obtained without re-compression.
- Constraints — deadlines, mandatory compatibility with the current system, requirements for the delivery format.
Describe the task and attach what you have: even an incomplete set already allows us to estimate the scope and propose a staged work plan. Then we will agree on the processing sequence, quality control points, and acceptance procedure using your own samples.