Engineers Propose ApsaraMQ LiteTopic Design to Stabilize High‑Concurrency Voice Agent Links

Que Xian, Wen Ting, Fu Li and Zhi Liu outline a message‑link redesign that uses the LiteTopic feature of ApsaraMQ for RocketMQ to stabilize real‑time speech interactions as voice agents scale. The authors argue that with ASR, TTS and LLMs reaching production maturity, the messaging link-not the models — becomes the primary bottleneck for large numbers of simultaneous users, so improving that link is critical to reliable deployment.

Their proposed architecture frames the message flow as APP → Gateway → BizProcessSystem (Route) → LLM/ASR/TTS and preserves persistent WebSocket connections between the APP and Gateway and between BizProcessSystem and LLM during interactions. The design emphasizes an end‑to‑end real‑time link that carries both upstream audio and downstream responses without breaking session continuity. The engineers highlight a practical shift from text to speech in agent scenarios — citing uses such as AI teachers, emotional companions and AI assistants — and note that speech enables more natural, multi‑round dialogue. That shift has driven real business adoption but also exposed scaling problems: many teams find messaging and session management fail long before model throughput becomes limiting.

They identify three core technical requirements for real‑time, high‑concurrency speech interactions. First, massive session management: systems must maintain tens or even hundreds of thousands of long connections simultaneously. Second, high‑frequency small‑packet transmission: clients slice audio into many small packets and the link must preserve continuity with minimal loss. Third, strict timeliness: latency sensitivity demands both high LLM throughput and a messaging link capable of real‑time notification and delivery.

The post details how traditional messaging architectures struggle under these combined demands, starting with precision routing for end‑to‑end session stickiness. Upstream audio and downstream feedback must be routed to the exact gateway node and backend processing instance the client is bound to; misrouting can drop audio streams or prevent asynchronous results from reaching the correct connection. Systems that perform well at low concurrency commonly break when required to enforce strict stickiness, high packet rates and low latency simultaneously.

For builders, the practical implication is clear: prioritize link design and messaging features that guarantee session stickiness, support high‑throughput small‑packet transport, enable asynchronous result push‑back, and provide explicit session lifecycle management. The authors position LiteTopic as a concrete messaging feature that implements these capabilities and offer architectural best practices to help teams move voice agents from prototype into high‑concurrency production.