Acoustic echo can make a voice agent hear its own playback, interrupt itself, or send a delayed copy of the far end back into the call. Production control depends on the right render reference, accurate timing, stable adaptation, and tests that cover real devices and double-talk.
What acoustic echo cancellation is
Acoustic echo cancellation (AEC) estimates the sound traveling from a local loudspeaker to a local microphone, then subtracts that estimate from the captured audio. It reduces the far-end audio that would otherwise be sent back to its source while preserving near-end speech for a full-duplex conversation.
Direction matters. Suppose a voice agent plays speech through a user's laptop speaker. The same laptop microphone captures the user, room noise, and a transformed copy of the agent's speech. Without effective AEC, the incoming stream can contain the agent's own playback. The user may hear an echo, and the agent's voice activity detection (VAD) may treat the returned playback as a user interruption.
Network or hybrid echo is a separate return path created in network or telephony circuitry, often around a two-wire-to-four-wire hybrid. Acoustic echo is local loudspeaker playback recaptured by a microphone.
A useful conceptual model is:
capture(t) = near-end speech(t) + echo(render audio, path, delay) + other noise(t)
The echo term includes much more than sound traveling across a room. Playback and capture buffers, converters, device processing, separate clocks, the loudspeaker, the room, and the microphone all shape its level and timing.
When you build on Dasha, keep waveform-level AEC at the endpoint that owns the physical playback and capture paths. Our acoustic echo detection can keep detected playback echo from causing spurious VAD interruptions. That control solves a related runtime problem and does not replace endpoint AEC.
How acoustic echo cancellation works
An AEC needs two synchronized inputs: the microphone capture and a copy of the audio sent toward the local loudspeaker, called the render reference. Processing then follows five broad stages.
- Feed the render reference into a delay buffer.
- Align the reference with the copy arriving in the microphone capture.
- Use an adaptive filter to model how the playback path transforms that reference.
- Subtract the predicted echo from the capture.
- Estimate and suppress residual echo that the linear model missed.
Markers 1 through 5 follow the workflow above: render-reference buffering, delay alignment, adaptive filtering, subtraction, and residual-echo estimation and suppression.
The render reference must match what was played
The reference is the known audio sent toward the speaker. For a voice agent, it usually includes the agent's synthesized speech. It must also include any other local playback that can leak into the microphone if you expect the canceller to remove it.
Tap the reference close to the actual render path and before the audio hardware. Preserve its relationship with user volume changes and other processing that affects playback. A reference taken before a source is mixed cannot predict that source. A reference that misses a later gain change gives the filter the wrong scale. Dynamic compression, limiting, speaker distortion, or operating-system processing after the tap introduces changes that a linear filter may model poorly.
This placement requirement explains why forwarding the agent's text-to-speech waveform to a remote server is rarely enough. The server does not automatically see the exact samples, buffering, volume, routing, and device processing that produced the acoustic signal at the user's microphone.
Delay alignment covers the complete device path
The canceller compares the recent render history with the capture to find the corresponding echo. The required delay covers:
- render queuing and buffering;
- digital-to-analog and analog-to-digital conversion;
- loudspeaker-to-microphone propagation and reflections;
- capture buffering and scheduling;
- resampling between different playback and capture clocks; and
- route-specific processing, including Bluetooth buffering.
The WebRTC Audio Processing Module (APM), for example, defines delay as the time between passing a far-end frame to render processing and passing the near-end frame containing its echo to capture processing. Its API accounts for both hardware rendering and capture-to-processing time. The WebRTC processing interface also places the module close to the hardware abstraction layer.
Delay is a tracked state, not a fixed room constant. A route switch, a resized jitter buffer, a busy audio thread, or a small sample-rate mismatch can move the echo outside the filter's useful alignment. WebRTC audio requirements explicitly call out different capture and playback clocks as a condition endpoint AECs should handle.
The adaptive filter learns the echo path
After alignment, an adaptive finite impulse response filter predicts the echo created by the render reference. Its coefficients represent the loudspeaker-to-microphone path over time. The filter updates those coefficients as it observes the difference between its prediction and the captured signal.
The path changes whenever a device moves, speaker volume changes, a laptop lid closes, or the audio route switches. The filter must reconverge without turning near-end speech into part of the echo model.
That last condition is the double-talk problem. Double-talk means the near-end and far-end speak at the same time. If the filter adapts aggressively during this period, it can learn from wanted near-end speech and become unstable. A robust controller reduces or pauses adaptation when the observation is unreliable, then resumes when it has clean evidence about the echo path.
Implementations differ. WebRTC's AEC3 is one concrete design, not a universal browser or device implementation. Its source shows main and shadow filter paths with subtraction, a residual-echo estimator, and a suppression filter. The shadow filter provides an alternate path estimate that can help recover when the main filter no longer matches the route.
Subtraction needs residual-echo suppression
The linear filter output is a predicted echo waveform. Subtracting it from the capture produces the linear AEC output. Perfect cancellation would require an exact model and exact timing. Real paths include loudspeaker distortion, clipping, time-varying processing, and long reverberant tails, so some correlated energy remains.
A residual-echo estimator predicts how much echo is left in each frequency region. A suppressor attenuates likely residual echo before the signal continues to VAD, speech recognition, or the network. Stronger suppression can reduce audible echo and false interruptions. It can also make wanted speech sound thin, watery, or gated, especially during double-talk. Tune for preserved near-end speech, not maximum attenuation.
Where AEC belongs in a voice AI stack
AEC belongs where the system can observe both the microphone capture and the corresponding local render reference with useful timing. That point is usually the endpoint producing the acoustic coupling.
| Call leg or product surface | Correct control point | Engineering implication |
|---|---|---|
| Browser on laptop speakers | Browser or device audio layer | Request echo control on the capture track and test each supported browser, operating system, and route. |
| Mobile speakerphone | Mobile operating system, audio session, or device DSP | Use the voice-communication route and handle route changes, Bluetooth, handset, and speakerphone separately. |
| Kiosk, robot, or embedded device | DSP or audio pipeline near the hardware layer | Feed the exact local render stream into AEC before post-AEC capture reaches VAD or speech recognition. |
| Headset | Endpoint route, often with AEC reduced or disabled | Acoustic coupling is usually low. Extra nonlinear processing can degrade clean headset speech. |
| Caller on a telephone network | The caller's handset, speakerphone, or relevant network endpoint | A cloud voice-agent service does not own the caller's physical speaker and microphone path. |
| Analog hybrid or gateway echo | Near the electrical hybrid or network gateway | Use network echo control designed for that path. ITU-T G.168 covers digital network echo cancellers, not room acoustics. |
The open-source WebRTC APM illustrates the endpoint pattern. It accepts a capture stream and a reverse-direction render stream and recommends placement close to the audio hardware. Server-side processing usually sees incoming participant streams without the remote device's render path.
A server can detect correlation between outgoing agent speech and returning audio, and it may suppress likely echo or protect interruption logic. It still lacks important endpoint state unless the application controls and synchronizes that endpoint. Treat server-side echo detection as a second line of defense.
For browsers, echoCancellation: true is a media constraint. The Media Capture specification defines the constraint and the reported track setting, while leaving the algorithm to the user agent and device. A successful request or a true setting confirms neither AEC3 nor an echo-free result. Check the applied setting, then run audio tests on the routes you support.
AEC and related audio controls solve different problems
Several controls can improve the same call while operating on different evidence.
| Control | Primary job | What it cannot prove or replace |
|---|---|---|
| Acoustic echo cancellation | Model local loudspeaker-to-microphone coupling from a render reference | Noise removal, endpoint routing correctness, or network hybrid echo control |
| Echo suppression | Attenuate audio likely to be echo, often after linear cancellation | Accurate waveform cancellation or clean double-talk by itself |
| Noise suppression | Reduce stationary or non-speech background noise | Removal of a known playback signal without a usable reference |
| Dereverberation | Reduce room reflections on wanted speech | Cancellation of far-end playback recaptured by the microphone |
| Automatic gain control | Keep capture level in a usable range | Recovery from clipping or a wrong render reference |
| Voice activity detection | Estimate whether captured frames contain speech | Distinguishing the user from correlated playback in every condition |
| Interruption handling | Decide whether detected user speech should stop agent playback | Cleaning the waveform sent to speech recognition or the far end |
AEC usually runs before VAD and speech recognition so those components see less returned playback. Noise suppression and gain control may share the same audio-processing module, but their order and interaction need route-level testing. Read our guides to voice activity detection and speech enhancement for the adjacent controls.
Why acoustic echo cancellation fails
Most production failures point to a mismatch in reference, timing, path behavior, or suppression policy.
| Failure mode | Typical symptom | Diagnostic direction |
|---|---|---|
| Missing reference | Full-level echo during one playback source | Confirm every audible source reaches the render input in the correct channel. |
| Wrong or duplicated reference | Poor convergence, pumping, or cancellation of unrelated audio | Compare reference samples and gain with the signal sent toward the device. |
| Delay error or variable buffering | Bursty or periodic residual echo | Plot delay estimates with route, buffer, and scheduling events. |
| Capture/render clock drift | Cancellation decays during a long call | Track delay movement and resampling behavior over several minutes. |
| Echo-path change | Short echo burst after moving the device or switching routes | Mark the change event and measure reconvergence. |
| Double-talk instability | Near-end words disappear or the filter diverges | Separate double-talk windows from far-end-only adaptation windows. |
| Loudspeaker nonlinearity | Harmonic or buzzy residual related to playback | Repeat at lower volume and bypass dynamic playback processing. |
| Capture or playback clipping | Harsh residual that persists despite alignment | Inspect pre-AEC peaks and saturation indicators. Lower gain before the clipping point. |
| Long reverberant tail | Late, diffuse echo remains | Test a longer modeled path or improve endpoint acoustics and placement. |
| Over-aggressive suppression | Metallic, watery, or missing near-end syllables | Compare linear output with final output during double-talk. |
| Cascaded cancellers | Pumping and unpredictable speech damage | Establish which layer owns AEC and disable redundant processing where the platform permits. |
Avoid debugging AEC through transcripts alone. A speech recognizer can hide a mild artifact, hallucinate through a severe one, or make a separate endpoint issue look like a language-model error. Preserve stage-level audio and timing evidence whenever privacy, consent, and retention policy permit it.
Production diagnostics that explain the failure
Capture three signals for a controlled reproduction when your stack exposes them:
- Render reference: the samples supplied to the canceller.
- Pre-AEC capture: microphone audio before echo processing.
- Post-AEC output: audio delivered to the next stage.
Also record sample rate, channel layout, route, input and output device identifiers, applied browser constraints, volume changes, buffer depth, AEC reset events, VAD decisions, interruption events, and playback start and stop times. These signals let you distinguish acoustic leakage from a duplicated software stream.
Where available, the following diagnostics help. The WebRTC APM statistics interface is one implementation that exposes them.
| Diagnostic | Meaning | Safe interpretation |
|---|---|---|
| Echo return loss (ERL) | Render power relative to echo power before cancellation | Describes acoustic coupling during valid far-end activity. Higher values usually mean less echo entered the microphone. |
| Echo return loss enhancement (ERLE) | Echo power before cancellation relative to output power | Describes canceller attenuation in suitable far-end-only windows. Near-end speech can make it misleading. |
| Delay median and variation | Estimated render-to-capture alignment and its spread | Sudden movement points to route or buffer changes. Persistent variation can indicate jitter or clock mismatch. |
| Divergent-filter fraction | Share of an aggregation window in which the linear filter is divergent | A rising fraction suggests adaptation or alignment trouble. |
| Residual-echo likelihood | Probability-like estimate that echo remains | Correlate peaks with audio, playback, and interruption events. It is implementation-specific. |
| Saturation or clipping flags | Whether the observed path exceeded its linear range | Alignment cannot repair samples already clipped. |
None of these is a universal call-quality score. ERLE can look excellent if a suppressor mutes everything. It can look poor during double-talk because wanted near-end speech remains in the output. Compute and compare metrics within labeled states such as far-end-only, near-end-only, double-talk, and silence.
Listening remains part of the release gate. Review the far-end return, the post-AEC signal, and the agent's interruption timeline together. Listen for returned words, chopped near-end onsets, musical artifacts, level pumping, and the time needed to recover after a route change.
A practical AEC test matrix for voice agents
Run the same deterministic prompts, levels, and timing across every supported endpoint class. Include real devices alongside lab loops because device DSP and routing often dominate the result.
| Scenario | Controlled stimulus | Expected behavior and evidence |
|---|---|---|
| Far-end-only speech | Play agent speech while the near end stays silent. Sweep normal playback levels. | Returned speech is reduced after convergence. Track residual audio, ERLE in valid windows, and false VAD or interruption events. |
| Near-end-only speech | Keep render silent while a user speaks at several levels and distances. | Speech stays intelligible with intact onsets and endings. Suppression does not gate or color clean capture. |
| Double-talk | Start user speech before, during, and after agent words. Include short acknowledgments. | User speech survives, interruption timing remains correct, and the filter stays stable after overlap ends. |
| Call startup | Begin render immediately, then after a quiet lead-in. | Record initial echo, time to useful convergence, and any startup false interruption. |
| Path change and reconvergence | Move the device, rotate it, close a laptop lid, or switch speaker volume mid-utterance. | Any echo burst is bounded, the delay estimate settles, and the filter reconverges without lasting speech damage. |
| Route matrix | Test headset, handset, speakerphone, Bluetooth, built-in laptop audio, and supported browser/OS pairs. | Each route uses the intended processing mode. Applied settings, artifacts, and interruption rates stay within route-specific baselines. |
| Clock drift and variable buffering | Run long calls while varying audio load or buffer depth in a test endpoint. | Alignment tracks gradual drift. Residual echo does not grow with duration. |
| Clipping and saturation | Raise render or capture gain through the onset of clipping. | Monitoring flags the condition. The system avoids interpreting harsh residuals as user barge-in. |
| Reverberant and nonlinear path | Use a reflective room, high speaker level, and any production playback processing. | Residual tails and speech damage remain within the approved listening baseline. Failures are attributable to a recorded path condition. |
| Playback-active false interruption | Run representative agent TTS with no user speech, including pauses and high-energy consonants. | Agent playback does not trigger user-speech interruption. Correlate every VAD event with residual-echo evidence. |
| Recovery from double-talk | Follow overlap with a far-end-only segment. | The filter returns to its prior attenuation range without an extended echo burst. |
Do not create one global pass threshold for every device and room. Establish baselines per supported route, then set release criteria around user impact and regression. A production gate can require:
- no reproducible self-interruption in playback-only tests;
- preserved user speech and acceptable transcription during double-talk;
- bounded reconvergence after supported route and volume changes;
- no new clipping, pumping, or musical artifacts in listening review;
- stable delay and filter behavior during long calls; and
- traceable audio, route, metric, VAD, and interruption evidence for every failed case.
AEC adds processing and buffering to the live audio path, so include it in the end-to-end voice AI latency budget. A change that reduces echo while making barge-in slow or damaging near-end speech is still a regression.
Production checklist
- Place AEC at the endpoint that owns microphone capture and local playback.
- Feed it the complete, correctly scaled render reference.
- Track delay across buffers, hardware, route changes, and separate clocks.
- Protect adaptation during near-end speech and double-talk.
- Detect clipping and nonlinear playback before tuning the filter harder.
- Treat residual suppression as a speech-quality tradeoff.
- Record route and configuration state with audio diagnostics.
- Segment metrics by talk state and correlate them with listening results.
- Test startup, drift, device changes, double-talk, and false interruptions.
- Keep one layer responsible for waveform-level cancellation.
How Dasha handles the adjacent interruption problem
Our runtime includes acoustic echo detection, which monitors calls for microphone and speaker feedback loops that can trip VAD. Per-call controls include vad_suppress_on_echo, echo_max_delay_seconds, and echo_coherence_threshold.
This feature protects turn-taking when returning playback resembles user speech. It is separate from waveform-level AEC. We do not present it as a replacement for the browser, handset, speakerphone, or embedded device canceller, and it does not guarantee an echo-free call.
If your team is building a production voice agent and wants managed telephony, turn-taking, testing, and call operations around that endpoint audio path, start a technical evaluation of Dasha.
