AI Noise Cancellation for Calls in Mobile Apps
When you open a microphone via AVAudioSession on iOS or AudioRecord on Android, you get raw PCM stream—everything around the user. Construction work outside, coffee machine, children—all goes into the call. Standard Acoustic Echo Cancellation (AEC) from WebRTC removes echo but not background noise. ML-based noise suppression model is needed here.
How AI Noise Cancellation Differs from Standard DSP
Classical approach—spectral subtraction or Wiener filter: noise model estimated during speech pauses, then subtracted from spectrum. Works for stationary noise (fan hum), fails on non-stationary (voice in subway, keyboard next to you).
AI approach—neural network trained on "clean speech + noisy speech" pairs predicts a mask for each spectrogram frame. Models like RNNoise, DTLN, or DistilledRNNoise run real-time, processing 10–20 ms frames with latency under one frame.
RNNoise: Quick Start on Both Platforms
RNNoise from Mozilla—C library, 90 KB, ~2 MFLOPS per frame. Compiles to static library for iOS (xcframework) and Android (AAR with native part via NDK).
// Initialize
DenoiseState *st = rnnoise_create(NULL);
// Process frame (480 samples = 10 ms at 48 kHz)
float frame[480];
// ... fill from microphone buffer
float vad_prob = rnnoise_process_frame(st, frame, frame);
// frame now contains denoised signal
// vad_prob > 0.5—likely speech
iOS integration: AVAudioEngine with custom AVAudioSinkNode or tap on input node. Format—Float32, 48 kHz, mono. AVAudioSession should be configured with mode: .voiceChat and explicitly disable system processing, otherwise iOS applies its own noise reduction on top of yours.
let inputNode = audioEngine.inputNode
let format = inputNode.outputFormat(forBus: 0)
inputNode.installTap(onBus: 0, bufferSize: 480, format: format) { [weak self] buffer, _ in
guard let self = self else { return }
let channelData = buffer.floatChannelData![0]
// Pass to rnnoise_process_frame via C-bridge
self.rnnoiseProcessor.process(channelData, frameLength: Int(buffer.frameLength))
}
On Android—AudioRecord with AudioFormat.ENCODING_PCM_FLOAT, buffer size 480 samples, processing in separate thread with Process.THREAD_PRIORITY_URGENT_AUDIO. Call the same C library via JNI.
DTLN and Heavier Models
If RNNoise insufficient (complex multi-component noise, multiple sources), use DTLN—dual-stage LSTM model. Converts to TFLite (Android) or Core ML (iOS).
In practice: DTLN at 16 kHz takes 8–14 ms per frame on iPhone 12, fits in real-time. Android Snapdragon 778G—similarly. On budget Helio G85—25–35 ms, creating cumulative latency.
For mobile, sample rate choice matters: 16 kHz instead of 48 kHz reduces computational load 4x, and for speech, bandwidth up to 8 kHz is sufficient for intelligibility.
Integration in WebRTC
WebRTC SDKs (LiveKit, Agora, Daily) provide AudioProcessingModule or hook before encoding. In native WebRTC for iOS—custom RTCAudioProcessingModule:
// Register custom processing
let config = RTCConfiguration()
// Create RTCPeerConnectionFactory with custom AudioDeviceModule
// or use AudioProcessingConfig for WebRTC built-in replacement
Important nuance: WebRTC already contains AECM and NS (Noise Suppression). When enabling your own AI noise cancellation, disable built-in NS via AudioProcessingConfig, otherwise double processing creates artifacts—"metallic" sound, clipped consonants.
Process
Audit app's audio pipeline, choose model per requirements (latency vs quality), compile native library for target architectures (arm64, x86_64 for simulator), integrate into existing audio stack, test on real noise.
Timeline Estimates
RNNoise integration into existing WebRTC stack takes 1–2 weeks. Implementation with DTLN/TFLite, VAD tuning, both platform support requires 3–5 weeks.