Build Production-Ready Speech-to-Speech AI Agents on AWS
Recording, recap, and the questions worth keeping – everything from Build Production-Ready Speech-to-Speech AI Agents on AWS in one place.
- Recorded 11 Jun 2026
- Presenters Jano Barnard & Jonas Jovaišas
- Stack Nova 2 Sonic · Nova Pro · Bedrock AgentCore
Watch it back
Get the slides
Session digest
If you missed it – or just want the gist before watching back.
Why voice
Voice is becoming a first-class way to talk to agents – real-time conversation that reads pace and hesitation, not just words. This session built one end to end on AWS, all the way to production.
Meet vox-brief
vox-brief is a one-click product-feedback voice interviewer, a persona called Alex. It returns a live transcript and a post-call digest – on call 24/7, scaling to any number of calls, and costing nothing when idle.
Prompting is the product
In the demo, one agent ran as three personas off near-identical prompts – the system prompt is where your secret sauce lives. The flow is rigid by default, but yours to branch, personalise, or loosen.
Serverless under the hood
Fully serverless: AgentCore Runtime hosts the agent, Nova 2 Sonic handles the voice, Nova Pro writes the digest, with CloudFront, API Gateway, Lambda, S3 and IAM around them. It scales to zero, so idle time costs nothing.
Production-ready
The real work is production-readiness – security, cost and reliability: WAF and CORS, per-user limits and token budgets, retries and a region strategy. AgentCore carries most of it, bundling identity, memory, gateway, observability, evaluations and policies.
Try it yourself
Talk to Alex, fork the code, go deeper.
Meet the Presenters
Learn from AWS experts working daily with high-growth startups
Jonas Jovaišas
Senior Technical Product Manager
Jano Barnard
R&D Engineer
Capabilities & cost
What the agent can do, where the limits are, and what it costs to run.
Not out of the box. Nova 2 Sonic is built for a one-to-one conversation – everything arriving on the mic is treated as a single “user”, and turn-taking is reactive: the model waits until it detects you’ve finished, then replies. It doesn’t identify who is speaking, and it won’t autonomously decide to interrupt an ongoing human-to-human exchange.
To build the experience described, you’d layer three things on top of Sonic:
- Speaker diarization / voice ID – a separate step to label who’s talking, since Sonic doesn’t do this natively. Overlapping speech is the genuinely hard part.
- An orchestration layer that owns the “should I jump in now?” decision and feeds only the relevant turns into Sonic.
- Per-speaker context – AWS’s own Nova 2 Sonic conversational-podcast sample does a version of this: a fresh stream per speaker turn with speaker-specific variables, rather than one stream listening to a whole room. Good reference pattern.
So a moderated multi-party setup is buildable, but it’s app-level engineering around the model, not a feature you flip on. For vox-brief’s 1:1 interview, single-speaker is exactly what you want anyway.
Full duplex / barge-in – yes, in the way that matters. The bidirectional streaming API (InvokeModelWithBidirectionalStream) holds one channel open for the call, audio flows both ways continuously, and the user can cut in mid-response. Sonic handles the interruption server-side; in vox-brief the browser also stops playback instantly on a barge-in and truncates the agent’s last transcript line so the UI matches what was actually heard. One operational caveat: a single Sonic connection is capped at 8 minutes by default – for longer calls you renew the connection and carry the prior conversation history forward (AWS ships a session-continuation pattern in the code samples).
VAD / pause detection – handled by Sonic, and tunable. You don’t wire up your own VAD; you stream raw audio and let the model decide when a turn ended. Nova 2 added turn-taking controllability, so you set pause sensitivity to low (patient – good when people pause to think), medium, or high (snappy – good for rapid back-and-forth). If you want client-side control instead, the frameworks Sonic now integrates with – Pipecat, LiveKit – bring their own VAD.
AEC – that’s not Sonic’s job, it’s the client’s. In vox-brief it’s the browser doing it: getUserMedia({ audio: true }) turns on the browser’s built-in echo cancellation, noise suppression and auto-gain by default, so the agent’s own voice coming out of the speakers doesn’t get re-fed into the mic. Move off the browser (telephony, native, headless) and you own AEC at that layer – Amazon Connect, Twilio, LiveKit and Pipecat all provide it. Sonic adds robustness to background noise, but it is not an echo canceller.
Roughly a cent per minute of live conversation – and zero when idle. The voice model dominates the bill; everything else is rounding error per call.
Per-call breakdown for a typical 5-minute interview (us-east-1, on-demand):
| Component | What drives it | ~ per 5-min call |
|---|---|---|
| Nova 2 Sonic | Voice in + out, ~25 speech tokens/s · $3 / $12 per 1M in/out | $0.045 |
| Nova Pro | One digest over the transcript at hang-up | < $0.01 |
| AgentCore Runtime | Active CPU + peak memory while the session is open | < $0.01 |
| API GW · Lambda · CloudFront | Per-request, on-demand | ~ $0.00 |
| All-in per call | idle cost = $0 | ≈ $0.05 |
A couple of things worth knowing: output speech is billed at 4× the input rate, so the agent talking is the main lever – keep its turns concise. AgentCore charges active CPU only (idle / I-O wait is free) plus peak memory for the session’s lifetime, which for a lightweight WebSocket proxy is sub-cent. The only real fixed cost is WAF, if you enable it – about $7/month flat + $0.60 per million requests (the --waf flag). Everything else scales to zero between calls.
These figures are us-east-1 on-demand at time of writing and will drift – treat them as ballpark, not a quote. Always confirm current rates against the official pages: Amazon Bedrock pricing (Nova 2 Sonic and Nova Pro tokens) and AgentCore pricing (runtime CPU and memory). Sonic is in-region only, so prices also vary by region. There’s a fuller teardown, including text-token caching tricks, in the companion blog post.
Building & running it
The follow-ups that usually come next.
Two Nova models, each doing what it’s best at:
- Nova 2 Sonic (
amazon.nova-2-sonic-v1:0) – the live voice conversation. Unified speech-to-speech, so there’s no separate ASR→LLM→TTS chain and no stacked latency. - Nova Pro (
amazon.nova-pro-v1:0) – the post-call digest. Structured JSON reasoning over the transcript: summary, key takeaways, action items, danger signals, sentiment.
Why Nova Pro (gen-1) for the digest rather than Nova 2 Pro? Nova 2 Pro is in preview and Forge-gated; Nova Pro is GA, predictably priced, and well-tested on reasoning. The digest runs after the call, so a couple of extra seconds is fine – stability beats novelty. Swapping in Nova 2 Pro later is a one-line change.
Every component scales to zero between calls. AgentCore Runtime spins up a per-session microVM when a call starts and shuts it down after; Lambda and CloudFront are pay-per-use; S3 just holds the static UI. Nothing runs – or bills – while no one’s on a call. The one exception is WAF if you turn it on, which carries a small flat monthly fee regardless of traffic.
AgentCore Runtime is serverless, so after a period of inactivity the microVM hosting the agent shuts down (idle window is configurable – default 15 minutes, up to 8 hours). The next call then pays a cold start while the container boots: anywhere from milliseconds to ~5 seconds, depending on image size and init work. Keeping the container lean is the main lever; for a demo, a quick warm-up call before you go live hides it entirely.
Nova 2 Sonic is in-region only – it doesn’t support cross-region inference profiles, because bouncing live audio between regions would wreck latency. As of writing it’s natively available in just three regions: US East (N. Virginia), US West (Oregon) and Asia Pacific (Tokyo). A Europe (Stockholm) launch was announced at re:Invent 2025 but then rolled back, so don’t assume an EU region is live – check the current list on the AWS model card before you plan around it.
That has a real consequence for EU data residency: with no EU region for Sonic and no cross-region inference to fall back on, there’s currently no way to keep the live audio inside the EU. If strict residency is a hard requirement, that’s a genuine constraint to design around today – pick the nearest available region and weigh the trade-off, keep the post-call digest (Nova Pro, which does run in EU regions) separate, or wait for an EU Sonic region to land. Don’t quietly route EU voice traffic to us-east-1 and call it solved.
Layered, as covered in the production-readiness section:
- Edge – WAF for rate limiting (vox-brief sets 300 requests / 5 min per IP) plus AWS managed rules and DDoS protection, and CORS so only your own front-end can talk to the agent.
- Identity – the demo uses basic HTTP auth via API Gateway + a presign Lambda and SigV4-signed WebSocket URLs. In production, AgentCore Identity (OAuth via your IdP, or Cognito) is the real answer.
- Prompt injection – a tight system prompt is your first line, since it’s prepended to every turn; AgentCore policies (Cedar) constrain which tools the agent may call. Then assume someone gets through anyway and apply least-privilege IAM to every component – never give the agent more than it needs.
Yes – that’s where AgentCore earns its keep. AgentCore Memory gives you short-term (a rolling window of recent turns) and long-term (knowledge-base-backed) context without you building the extraction or retrieval pipeline. AgentCore Gateway wraps existing APIs, Lambda functions, or MCP servers as tools the agent can call. vox-brief doesn’t use memory – a one-shot interview doesn’t need it – but both are a small lift to add. Nova 2 Sonic also supports asynchronous tool calling, so the agent can run a lookup mid-conversation without falling into dead air.
Nova 2 Sonic speaks seven languages natively: English, French, Italian, German, Spanish, Portuguese and Hindi – with English covering US, UK, Indian and Australian accents. Polyglot voices let a single voice switch languages mid-session and still sound natural, and it does automatic language detection and switching, so you’re not locking the session to one language up front.