How Libopus Manages FEC to Mitigate Packet Loss

Real-time audio transmission over the internet is highly susceptible to packet loss, which causes audio dropouts and severe degradation in voice quality. To combat this, the Opus audio codec, implemented via the libopus library, utilizes in-band Forward Error Correction (FEC). This article explains how libopus dynamically manages FEC, embeds redundant low-bitrate audio data within subsequent packets, and decodes this information on the receiving end to reconstruct lost packets and maintain seamless, high-quality audio communication.

Understanding In-Band FEC in Opus

Unlike traditional out-of-band FEC, which transmits redundant repair data in separate network packets, libopus uses an “in-band” approach. In-band FEC embeds a highly compressed, lower-bitrate copy of the current audio frame (Frame \(N\)) inside the payload of the immediately following audio packet (Frame \(N+1\)).

If Frame \(N\) is lost during transmission but Frame \(N+1\) arrives successfully, the decoder can extract the redundant data from Frame \(N+1\) and reconstruct the missing audio. This approach minimizes network overhead and prevents the packet multiplication associated with out-of-band FEC.

Dynamic Control and Network Feedback

The libopus encoder does not apply FEC blindly. Since sending redundant data increases the overall bitrate, the library dynamically manages FEC based on real-time network conditions.

1. Packet Loss Rate (PLR) Monitoring: The application hosting libopus must monitor network packet loss (typically via RTCP feedback in VoIP applications) and pass this percentage to the encoder using the OPUS_SET_PACKET_LOSS_PERC control signal.
2. FEC Activation Thresholds: The encoder decides whether to insert FEC data based on the reported packet loss rate and the available bandwidth. If packet loss is low (e.g., below 1%), FEC is kept disabled to save bandwidth. As packet loss increases, the encoder automatically starts allocating a portion of the total bitrate to the redundant FEC data.
3. Bitrate Trade-off: When FEC is active, the encoder must split its target bitrate between the primary audio stream and the redundant stream. libopus intelligently manages this trade-off so that the primary audio quality is not overly degraded.

The Encoding and Recovery Process

The actual implementation of FEC within the library relies on the SILK voice-codec layer of Opus. The process operates through the following steps:

• Low Bit-Rate Redundancy (LBRR): When FEC is enabled, the encoder compresses the current frame twice. The first is the primary high-quality stream. The second is a highly optimized, low-bitrate representation known as the LBRR frame.
• Packetization: The encoder packages the primary data of the current frame alongside the LBRR data of the previous frame into a single packet payload.
• Decoding and Restoration: Upon receiving a packet, the decoder checks if the sequence number indicates that the previous packet was lost. If a gap is detected, the decoder checks the current packet’s payload for LBRR data. If FEC data is present, the decoder decodes it to fill the gap. If no FEC data is present, the decoder falls back to Packet Loss Concealment (PLC), which uses interpolation to estimate the missing sound.

Codec Modes and FEC Constraints

It is important to note that true in-band FEC in libopus is natively supported primarily within the SILK mode (optimized for voice and lower bitrates) and the Hybrid mode (which combines SILK and CELT).

The CELT mode (optimized for high-fidelity music and ultra-low latency) does not support SILK’s LBRR-based FEC. Instead, CELT relies on Packet Loss Concealment (PLC) or application-layer redundancy (such as sending duplicate packets) to handle packet loss. For VoIP and conferencing applications where packet loss is expected, configuring Opus to operate in SILK or Hybrid mode is essential to leverage libopus’s robust FEC management.