Real-Time Transport Protocol (RTP) carries live audio and video with sequence numbers and media timestamps. It does not guarantee delivery, low latency, or quality of service. For technical teams building voice AI, understanding that boundary helps separate call signaling, media delivery, and slow agent responses. The packet header shows what arrived; application events explain what the agent did.
What RTP does in a voice call
RTP gives a receiver the information needed to identify a media payload, follow packet order, and place media on a playback timeline. Its core specification is RFC 3550. The payload is encoded media, not a transcript or an instruction for an AI model.
A voice application divides audio into units, encodes them, and packages them into RTP packets. The receiving application reconstructs the stream and handles decoding and playout, the release of audio for listening or downstream processing. RTP's timing information supports that work. The receiver implements the buffering and playback policy.
We help technical teams build and run production voice AI agents through a managed runtime, APIs, and a web application. Our telephony workflows include Session Initiation Protocol (SIP) trunk credentials and per-call SIP signaling traces. Understanding RTP helps keep that signaling evidence separate from evidence about audio delivery.
SIP establishes the session; RTP carries the media
SIP creates, changes, and ends sessions. It does not carry the conversation's audio samples. That separation is part of the SIP specification.
A simplified one-way media path is:
- Audio source: Microphone input or generated speech supplies audio.
- Encoder and packetizer: The application encodes audio and adds RTP metadata.
- Network path: Packets travel over an underlying transport and network.
- Receiver: The application handles arrival order, decoding, and playout.
A successful session setup and usable audio are separate outcomes. A SIP exchange can establish a call while the media path has a problem. Conversely, audio can reach the receiver while the agent takes too long to decide what to say. Our SIP trunking guide covers the signaling and integration side.
RTP is commonly carried over UDP
User Datagram Protocol (UDP) supplies the datagram transport commonly used beneath RTP. RTP adds media-specific metadata. They occupy different roles in the same stack.
UDP is common, not mandatory. RTP can use other suitable underlying transports, including connection-oriented RTP framing over Transmission Control Protocol (TCP). The underlying transport affects delivery, ordering, and delay. RTP alone does not reserve bandwidth or guarantee quality of service (QoS).
Web Real-Time Communication (WebRTC) is a larger protocol suite. It uses RTP media transport with additional requirements, including Secure Real-Time Transport Protocol (SRTP). RTP and WebRTC are not interchangeable names. The WebRTC media specification defines that relationship; our WebRTC vs WebSocket guide explains the broader transport choice.
What an RTP packet contains
The fixed RTP header occupies 12 bytes. Optional contributing-source identifiers and a header extension can add to it. The payload follows the header; padding, when present, is excluded from the payload.
| Field | Meaning |
|---|---|
| Version, 2 bits | RTP version; the current format uses 2. |
| Padding and extension, 1 bit each | Flags for trailing padding and an additional header extension. |
| Contributing-source count, 4 bits | Number of contributing-source identifiers after the fixed header. |
| Marker, 1 bit | Profile-defined event marker. |
| Payload type, 7 bits | Identifier for the payload's interpretation. |
| Sequence number, 16 bits | Packet-order counter. |
| Timestamp, 32 bits | Media sampling-time value. |
| Synchronization source, 32 bits | Identifier for the packet stream's source, abbreviated SSRC. |
| Contributing sources, optional | Source identifiers in mixed media, abbreviated CSRCs. |
| Payload | Encoded media interpreted using the agreed payload format. |
The RTP header definition specifies these fields. Three distinctions matter when interpreting them:
- Sequence number versus timestamp: The sequence number tracks packets. The timestamp locates media in time.
- Source versus caller identity: The synchronization source (SSRC) identifies a stream's timing and sequence space. It is not an application-level customer or call identifier. Contributing sources (CSRCs) identify inputs to a mixed stream.
- Payload identifier versus codec selection: The payload type is meaningful within the applicable RTP profile and session mapping. Its value does not independently negotiate a codec, the algorithm used to encode and decode media.
An RTP profile supplies conventions such as payload-type assignments and marker behavior. A payload-format specification defines how a particular encoding is packaged. For example, the audio/video RTP profile defines static assignments and dynamic mappings. A dynamic payload number has no universal codec meaning without its mapping.
Sequence numbers and timestamps answer different questions
For a continuous fixed-duration audio stream, the sequence number advances once per sent packet. The RTP timestamp advances according to the payload's RTP clock rate and the amount of media represented. The clock rate must come from the applicable payload definition or session mapping.
Assume a stream uses an 8,000-tick-per-second RTP clock and each packet represents 20 milliseconds of audio. Each packet then advances the timestamp by:
8,000 × 0.020 = 160 ticks
This is an arithmetic example, not a Dasha configuration. The initial timestamp is arbitrary. Only differences within the same timing space are useful here.
Suppose three packets from the same SSRC reach a receiver in this order:
| Arrival order | Sequence | RTP timestamp |
|---|---|---|
| First | 5000 | 100000 |
| Second | 5002 | 100320 |
| Third | 5001 | 100160 |
The second arrival creates a temporary sequence gap. The third fills it. All three packets arrived, but they arrived out of order. Their media order is 5000, 5001, 5002, with each timestamp step representing 20 milliseconds under the stated clock assumption.
The receiver may be able to reorder them before playback. Whether packet 5001 remains useful depends on its arrival relative to the receiver's playout deadline. Packet presence and timely playback are different facts.
A timestamp gap also needs context. With silence suppression, a sender can stop sending audio packets while its media clock continues. The next packet can have a consecutive sequence number and a larger timestamp jump. The profile's silence-suppression rules explain why a media-time gap alone does not prove packet loss.
An RTP timestamp is not a latency measurement
The timestamp is a media-clock value, not the receiver's arrival time or a wall-clock reading. Subtracting it directly from a local system timestamp does not produce one-way network delay.
It also cannot measure the interval between a caller finishing a question and hearing an AI response. Those events span input delivery, turn detection, response generation, output delivery, and playback. They need application-level timing. Our voice AI latency guide explains those boundaries.
RTP and RTCP: media and delivery reports
RTP Control Protocol (RTCP) is RTP's companion reporting protocol. RTP packets carry media; RTCP reports describe reception and sender activity and support synchronization.
A receiver report includes reception information such as loss and interarrival jitter. A sender report includes sender counters and a relationship between RTP media time and wall-clock time. These reports provide delivery evidence. They do not make the network reliable or measure the whole agent response pipeline.
The report formats are defined in RTCP sender and receiver reports. A report's meaning depends on the stream and reporting endpoint. Healthy reception at one endpoint does not establish healthy reception everywhere along a bridged call path.
Loss, jitter, and recovery belong to different mechanisms
Loss means expected packets were not received at the observation point. Reordering means packets arrived in a different order from the sender's sequence. Jitter is variation in transit timing relative to the media timeline. These conditions can occur together, but they are different measurements.
A jitter buffer holds incoming media briefly to smooth arrival variation. More waiting gives delayed packets a better chance of making playback, while adding delay to the conversation. A packet received after its playback deadline can be unusable even though it was delivered. Wireshark's RTP analysis documentation distinguishes packet statistics from playback behavior, including jitter-buffer drops.
RTP supplies ordering and timing metadata. Base RTP does not automatically resend a missing packet. Additional mechanisms can provide feedback and recovery:
- The RTP profile defined in RFC 4585 supports RTCP feedback, including negative acknowledgments for missing packets.
- RFC 4588 specifies an RTP retransmission format. Recovery requires the corresponding endpoint support and session setup.
Detecting a gap, sending feedback, and receiving a useful replacement are separate steps. Recovery can consume more time than a conversational playout deadline allows. The retransmission specification explicitly accounts for reordering and whether a recovered packet will still be useful. A feedback feature's existence therefore does not establish that every missing audio packet will be repaired in time.
RTP is different from RTSP
Real-Time Streaming Protocol (RTSP) controls streaming sessions, including setup and playback commands. RTP transports media. A streaming system can use them together, but they are separate protocols with separate jobs. The RTSP specification defines that control role.
The distinction is the same practical one that matters with SIP: session control does not itself prove successful media delivery.
Production voice AI needs evidence at the right boundary
RTP is useful when a team needs to explain what happened to media between a sender and receiver. It cannot, by itself, explain every pause, missing word, or failed call.
Useful ownership is explicit:
- Session parameters: The signaling component owns the exchange of media formats and transport details. The payload mapping and RTP clock rate give header values their meaning.
- Reception evidence: Loss, arrival timing, and late-discard information belong to a specific receiver, stream, and direction. A call-wide average can hide a problem on one leg.
- Playback policy: The receiving media component owns buffering and the deadline after which a packet is no longer useful.
- Recovery: The endpoints and negotiated session determine which feedback or retransmission mechanisms apply.
- Agent response timing: Application events locate delays before output media exists. A healthy outgoing packet stream can begin after a long generation pause.
That separation avoids a common debugging mistake: treating a signaling trace, packet counter, and response-time metric as interchangeable evidence.
For signaling-side debugging in Dasha, our SIP trace API returns per-call SIP messages such as INVITE, ACK, and BYE, including their raw content. This is signaling evidence, not an RTP packet capture. Likewise, the transport field in our SIP credential configuration selects the SIP signaling transport; it does not select RTP media transport.
For a managed production voice AI integration, start with our SIP configuration guide to connect the signaling side of your telephony setup.
