Channels: the basic idea
Audio in a video file is organised into channels, and each channel is intended for a particular speaker position. Mono has one channel. Stereo has two, left and right, and it is still the default for most online video, because most people listen on phones, laptops or headphones.
Surround sound adds channels so that sound can seem to come from around the listener instead of only from the front. Whether that effect is noticeable depends heavily on the listening setup.
Stereo, 5.1 and 7.1
The label 5.1 describes six channels: front left, front centre, front right, surround left, surround right, and one low-frequency effects channel. The “.1” is that last channel, which carries only deep bass and is sent to a subwoofer. A 7.1 layout adds two more surround channels, giving eight in total.
The centre channel matters more than it might seem. In film and TV mixes, dialogue is commonly anchored there, which helps speech remain clear. Music and effects are spread across the other channels.
Surround mixes are channel-based: each channel is a fixed track, and a speaker layout must match it. If the listener has only two speakers, the player has to fold the channels down to stereo, a process called downmixing. Done well, it is barely noticed. Done poorly, dialogue can sound quiet or effects can overwhelm it, which is one reason a separate stereo mix is sometimes supplied.
Object-based and spatial audio
Newer systems, of which Dolby Atmos is a well-known example, add object-based audio. Instead of assigning every sound to a fixed channel, the mix includes sound objects with position data, such as a helicopter moving overhead. The playback system then renders them for whatever speakers are present, which can include ceiling speakers, a soundbar, or headphones using binaural processing.
For headphones, spatial audio generally relies on filters that imitate how sound reaches each ear. Results vary between listeners and products, and the effect is a simulation rather than a true room of speakers. Treat it as a pleasant extra, not a guarantee of a particular experience.
Audio formats used with video
Audio is compressed, much like picture, and the audio codec is stored inside the container alongside the video stream.
- AAC is very common in MP4 files and in online delivery, and often carries stereo or 5.1.
- Dolby Digital (AC-3) and its successor Dolby Digital Plus are widely used for broadcast, discs and streaming, including 5.1 and, in the Plus version, formats that carry object-based audio.
- DTS is another family of multichannel formats, found particularly on discs.
- Opus is an open codec used in WebM and in real-time communication, and it performs well at low bitrates.
- PCM is uncompressed audio, used in production and on some discs because nothing is discarded.
- FLAC is lossless compression, which is smaller than PCM yet fully reversible.
Most of the codecs above are lossy compression, which discards detail the ear is least likely to miss. Higher audio bitrate generally preserves more detail, although codecs differ in efficiency, so comparing numbers across different codecs can mislead.
How streaming services handle multichannel audio
Streaming services deal with an enormous range of devices, from phones with a single tiny speaker to home cinema systems. A common approach is to offer several audio tracks for the same programme, for example a stereo track and a 5.1 track, and let the player choose what suits the device and connection. The system first checks what the device and its connected equipment can decode, and the player then selects an appropriate track.
This is one part of the wider adaptive bitrate approach, and the choice of tracks is described in the manifest files used by HLS and DASH. Surround tracks need more data than stereo, so on weak connections a service may prefer the stereo version. Immersive formats usually require compatible apps, devices and sometimes specific cables or settings, which is why someone can own good equipment and still hear only stereo.
Language tracks, commentary and descriptive tracks are handled the same way, as separate audio streams chosen by the viewer. A descriptive track is the basis of audio description.
Practical points for creators
Most creators publishing online should deliver a clean stereo mix. It plays everywhere, and a surround mix that is not heard as intended brings few benefits. Good dialogue recording matters more than channel count, a point developed in the discussion of audio quality.
If a project does need surround, plan for it from the start. Mix in a suitable room, test the downmix to stereo, and confirm that the destination platform accepts the format. Loudness is another consideration: many platforms normalise volume, and the specific targets vary, so check the current guidance rather than relying on a remembered number.
When checking a finished file, play it on several systems, including phone speakers and headphones. A mix that sounds balanced on a good monitor setup can lose its dialogue on a small speaker. For mixing choices, see the notes on audio mixing, and for the equipment side, the guide to microphone types.