Case study/Real-time AI platform/2026
SeloraX Voice AI
A voice agent platform that answers real phone calls with a speech model trained in-house.
- Next.js
- TypeScript
- Node.js
- Custom speech model
- LiveKit
- WebRTC
- MongoDB
- S3
01The problem
Each client wanted its own phone agent: its own prompt, its own tools, its own knowledge, billed separately, answering real calls in real time.
Voice is unforgiving: a pause that feels short in a chat feels broken on a call, and callers interrupt. The platform had to stream audio both ways with low delay and let people cut the agent off mid-sentence.
02How it fits together
The gateway holds one persistent connection to the model for the whole call; everything else hangs off it.
03Decisions
- 01
One persistent WebSocket per call
Opening a connection per turn would add a handshake to every reply. Keeping one socket open for the whole call keeps latency down and keeps the call's state in one place.
- 02
Barge-in and live transcripts in the gateway
The gateway listens while the agent speaks, so a caller can interrupt and the agent stops, which is how real conversations work. Live transcripts come from the same stream.
- 03
Self-hosted LiveKit for WebRTC
Running the WebRTC media server ourselves keeps call audio inside our own infrastructure.
- 04
Money in poisha, never floats
Balances and per-customer markups are integers in the smallest currency unit, so billing never drifts from floating-point rounding.
- 05
Capture every turn to improve the model
Each turn's speech and text is stored, which feeds the training data for the next version of the voice model (see the Bangla Voice Model case study).
04Trade-offs
- A persistent socket per call is simple and fast, but each live call holds a connection and its state in memory, so capacity scales with concurrent calls.
- Self-hosting LiveKit means owning media infrastructure (updates, scaling, monitoring) in exchange for control over where audio goes.
05Results
- Answers real phone calls over SIP as well as browser microphone sessions.
- Each client gets its own agent: prompt, HTTP and client-side tools, lifecycle webhooks, a knowledge base, and S3 call recordings.
- Billing per customer with markup, precise to the poisha.
- A data loop from live calls back into model training.