A fintech company whose voice bot processed transfers faced fraud: fake voice commands generated via WaveNet passed verification unchallenged. Losses hit 15% of transaction volume. They needed a speaker verification system with anti-spoofing capable of filtering synthesized audio in real time. Our stack — ECAPA-TDNN for embedding extraction and CQCC-LCNN for spoof detection.
Problems We Solve
The first is replay attacks: an attacker simply plays a recording. Text-dependent mode is helpless here — anti-spoofing is needed. The second is high voice variability due to colds, fatigue, or noise. Without an adaptive threshold, FRR can exceed 10%. The third is speed: the system must respond in <200 ms, or UX suffers. Moreover, synthesized voices based on WaveNet and Tacotron are becoming increasingly realistic, and traditional methods can't cope.
Attacks on Voice Systems
We distinguish three main types: replay (recording playback), synthesis (WaveNet, Tacotron), and conversion (voice transformation into another). Replay is blocked by adding nonce and timestamp to the request. Synthesis and conversion are detected by CQCC-LCNN trained on ASVspoof 2021 — 98% accuracy at 1% FAR. Replay attack protection reduces losses by up to 90%.
How We Select the Verification Threshold?
The threshold determines the balance between FAR (accepting an impostor) and FRR (rejecting a genuine user). For banking scenarios, FAR <0.5% is needed; for app authorization, 1% is sufficient. We tune the threshold to your scenario using the ROC curve on your data. The table below shows typical thresholds:
| Threshold | FAR | FRR | Application |
|---|---|---|---|
| 0.1 | 5% | 1% | Low risk (app authorization) |
| 0.25 | 1% | 5% | Balanced (normal scenarios) |
| 0.4 | 0.1% | 15% | High security (banks, payments) |
More about metrics
FAR (False Acceptance Rate) — the proportion of errors when the system accepts an impostor. FRR (False Rejection Rate) — the proportion when it rejects a genuine user. EER (Equal Error Rate) — the point where FAR and FRR intersect, a standard quality metric. The average EER in our deployments is 1.2%.Architecture comparison: ECAPA-TDNN gives EER 1.2x lower than x-vectors (0.87% vs 1.05% on VoxCeleb1). For resource-constrained scenarios, we use ResNetSE34L with INT8 quantization — inference on CPU in 50 ms.
| Architecture | EER (%) | Inference (GPU/CPU) | Model Size |
|---|---|---|---|
| ECAPA-TDNN | 0.87 | 80 ms / 200 ms | 20 MB |
| x-vectors | 1.05 | 60 ms / 150 ms | 15 MB |
| ResNetSE34L (INT8) | 1.10 | 30 ms / 50 ms | 5 MB |
Implementation on ECAPA-TDNN
We use a pretrained model from SpeechBrain: ECAPA-TDNN. It outputs embeddings in 192-dimensional space. Inference speed — 80 ms on GPU, 200 ms on CPU. Code:
from speechbrain.pretrained import SpeakerRecognition import torchaudio verifier = SpeakerRecognition.from_hparams( source="speechbrain/spkrec-ecapa-voxceleb", savedir="tmp_verification" ) def verify_speaker( enrollment_audio: str, test_audio: str, threshold: float = 0.25 ) -> tuple[bool, float]: """ enrollment_audio: reference recording of a registered user threshold: Accept/Reject threshold (tuned for needed FAR/FRR) """ score, prediction = verifier.verify_files(enrollment_audio, test_audio) is_same = float(score) >= threshold return is_same, float(score) Why Anti-Spoofing Is Needed?
Without it, the system is vulnerable: synthesized voice (WaveNet, Tacotron) passes verification. We add an additional classifier based on CQCC-LCNN that distinguishes recordings from live speech. It runs before the main comparison, blocking 98% of attacks. The average project cost with anti-spoofing is $5,000, and monthly savings from fraud prevention reach $15,000.
from speechbrain.pretrained import EncoderClassifier antispoofing = EncoderClassifier.from_hparams( source="speechbrain/asvspoof-cqcc-lcnn", savedir="tmp_antispoofing" ) def is_genuine(audio_path: str) -> bool: signal, _ = torchaudio.load(audio_path) prediction = antispoofing.classify_batch(signal) return prediction[3][0] == "genuine" Typical Implementation Mistakes
- Collecting only one enrollment phrase is bad. Use 3–5; averaging gives -30% EER.
- Not updating enrollments — voice changes. Re-record every 3–6 months.
- Ignoring replay — add nonce and timestamp to the request.
- Using default threshold — always calibrate on your data.
- Forgetting about noise — minimum SNR 15 dB, otherwise accuracy drops.
Implementation Process
- Analytics: gather requirements for FAR/FRR, attack types, integration points.
- Prototype: in 2 days set up the model, test on your recordings, tune threshold.
- Integration: embed into bot/application via REST API or gRPC.
- Load testing: verify p99 latency < 300 ms at 500 RPS.
- Deployment and monitoring: deploy on Kubernetes with autoscaling, log metrics.
What's Included in the Work?
- Documentation on architecture and API specification (OpenAPI).
- Docker image with the model (GPU/CPU version).
- Instructions for deployment and operation.
- Training your team (2–3 days).
- 6-month warranty on the model with possibility of fine-tuning.
We have been in voice biometrics for over 5 years, completed more than 30 projects for fintech and telecom. Average EER in our deployments is 1.2%.
Timelines
Basic system (verification + thresholds) — from 1 week. With anti-spoofing and profile management — 2–3 weeks. Cost depends on the number of instances and load. We estimate projects in 1 day.
Want to test speaker verification on your data? Order a pilot project — we'll adapt the model in 1 day. Get a consultation on your project — we'll send a preliminary estimate.