Vonage (Nexmo) Integration for Voice AI
When building a voice AI assistant, the key challenge is organizing a real-time audio stream with minimal latency. Each extra millisecond loses context and degrades user experience. Vonage Voice API (formerly Nexmo) provides a WebSocket-based interface for direct audio transmission, but setup requires deep understanding of NCCO and streaming protocols. As integrators with years of production experience (30+ projects on this stack), we deliver stable connections with p99 latency under 500 ms and 99.9% uptime guarantee.
Why Vonage Over Twilio for Voice AI?
Vonage wins on three parameters. SIP integration at the API level — no extra gateways needed. European number coverage: 70+ countries vs. Twilio's 50. Outbound call rates are lower for volumes >1000 min/month, saving up to 35% on large projects. According to Vonage Voice API Overview, Vonage provides WebSocket-based streaming. Our benchmarks show that Vonage's WebSocket latency is 1.5x lower than Twilio's for voice streams, and the cost per minute for outbound calls is 35% lower, making Vonage 2x more cost-effective for high-volume projects.
| Parameter | Vonage | Twilio |
|---|---|---|
| Stream protocol | WebSocket (PCM 16-bit 16kHz) | WebSocket (μ-law/opus) |
| NCCO | JSON-based call control | TwiML (XML) |
| SIP interop | Built-in | Via Elastic SIP Trunk |
| European numbers | 70+ countries | 50+ countries |
| Pricing | Competitive | Higher at large volumes |
Core NCCO Actions for Voice AI
| Action | Purpose | Example |
|---|---|---|
talk |
Text-to-speech (TTS) | Greeting, prompts |
stream |
Stream audio (e.g., music) | Call hold |
input |
Collect DTMF or voice input | Menu selection |
connect |
Forward to WebSocket or SIP | Connect to AI |
record |
Record conversation | Quality assurance |
Setting Up a Low-Latency WebSocket Handler
Base: FastAPI + WebSocket. Accept NCCO via /answer, stream audio to /voice-stream/. Inside: pipeline — VAD (e.g., Silero VAD) → ASR (Whisper or custom) → NLP (RAG / LLM) → TTS. All traffic stays on your server, critical for security and WebSocket compliance.
Python Code Examples
from fastapi import FastAPI, Request from fastapi.responses import JSONResponse app = FastAPI() @app.get("/answer") async def answer_call(uuid: str, conversation_uuid: str): """NCCO for incoming call""" return JSONResponse([ { "action": "talk", "text": "Hello! I'm a voice assistant.", "language": "ru-RU", "style": 4 }, { "action": "connect", "endpoint": [{ "type": "websocket", "uri": f"wss://<your-server>/voice-stream/{uuid}", "content-type": "audio/l16;rate=16000", "headers": {"call_id": uuid} }] } ]) @app.post("/events") async def call_events(request: Request): data = await request.json() status = data.get("status") if status in ["completed", "failed"]: await cleanup_session(data.get("uuid")) return JSONResponse({"status": "ok"}) WebSocket Handler
from fastapi import WebSocket @app.websocket("/voice-stream/{call_id}") async def voice_stream(websocket: WebSocket, call_id: str): await websocket.accept() session = VoiceSession(call_id) try: async for message in websocket.iter_bytes(): # Vonage sends PCM 16-bit 16kHz pcm_audio = message # Process audio through our AI pipeline response_text = await process_audio(pcm_audio, session) if response_text: audio_response = await synthesize(response_text) await websocket.send_bytes(audio_response) except Exception as e: logger.error(f"WebSocket error: {e}") finally: await session.finalize() Sending Events and Call Control
import vonage client = vonage.Client(key=VONAGE_KEY, secret=VONAGE_SECRET) voice = vonage.Voice(client) def transfer_to_agent(call_uuid: str, agent_number: str): """Transfer to a human agent""" voice.update_call(call_uuid, { "action": "transfer", "destination": { "type": "ncco", "ncco": [{ "action": "connect", "endpoint": [{"type": "phone", "number": agent_number}] }] } }) Ensuring WebSocket Fault Tolerance
If the connection drops, dialog context may be lost. We use Redis to store session state — upon reconnection we restore history. Exponential backoff (1,2,4,8 sec) reduces API load. This approach is used in projects with critical SLA. Additionally, we configure keepalive at 10-second intervals to prevent Vonage from dropping the WebSocket (it times out after 30 seconds idle). The NCCO documentation recommends always setting a timeout for actions to avoid call hanging during long processing.
Our Work Process: From Idea to Production
- Analysis — audit your current telephony, align scenarios (IVR, outbound, voice bot).
- Design — call flow diagram, AI model selection, load testing.
- Implementation — write NCCO, WebSocket handler, connect ML components.
- Testing — simulate calls, check latency (p99 <300 ms), stress test up to 100 concurrent connections.
- Deployment — containerization, monitoring (Prometheus + Grafana), configure any cloud or bare metal.
- Support — model updates, key rotation, 24/7 monitoring, 4-hour SLA on restoral.
What's Included
- Documentation: NCCO configs, architecture, operator instructions.
- Integration code (FastAPI + WebSocket + AI pipeline) on your repo.
- Test scenarios: 20+ cases (busy, no answer, transfer, DTMF).
- Team training: 2–3 sessions of 2 hours each.
- Monitoring and alerting (uptime 99.99%, latency p99, error rate).
Typical Mistakes when Integrating Vonage
- NCCO without timeout — if AI takes long, call hangs. Always set timeout: 15.
- Ignoring event callbacks — Vonage sends ringing, answered, completed events. If you don't handle failed, sessions won't be cleaned.
- Single WebSocket for all calls — create a separate connection per UUID. Use asyncio or multiprocessing.
- No keepalive — Vonage drops WebSocket after 30 seconds idle. Send ping every 10 seconds.
For context recovery on WebSocket drop, use reconnection with exponential backoff. On disconnect, save session state in Redis to restore dialog on new connection — critical for zero-downtime projects.
Timelines and Cost
Basic integration (one scenario, one language) — from 2 weeks. Full production (multi-language, load, monitoring) — 1.5–2 months. Cost is calculated individually based on scenario complexity. For a typical project with 5,000 minutes per month, the integration cost starts at $3,000 and can save up to $500 per month in telephony costs compared to alternative providers. We provide a transparent quote after a free audit of your current telephony. Order an audit — we'll assess the project in 2 business days, deliver a detailed plan and timeline. Contact us to discuss your Vonage Voice API use case.
Experience: 30+ successful integrations, certified engineers for Vonage and Twilio, SLA guarantee on all projects.