The Most Common Video Formats and Codecs Explained

Digital video is all around us. Whether it’s a disc, a streaming service, or a file on your computer, every video has a specific format. Understanding common video formats is essential both for people who make video and people who simply want to watch video.

Otherwise you might put out content with poor quality or just won’t know why a given video won’t play for you. In this article we’ll explain what video formats are as well as the most common ones everyone should know.

Format, Container And Codec: Key Concepts Explained

The word “format” when applied to videos needs a bit of unpacking. In any medium, the format is its standardized form. VHS and Betamax were formats for home video tape. While both used the same basic technology (TV signals recorded on magnetic tape), the exact method and design differed between the two.

The end result is that a VHS tape will not work (or even fit) into a Betamax machine and vice versa. Digital video is no different. There are many different ways of encoding video and audio as digital data. So a player cannot understand or play back a format it was not designed to.

For a digital video, the format refers to the sum total of all the bits and pieces that come together into the final video file. The first thing you’ll see is the container. That is, whether the file is an .AVI. .MOV, .MP4 and so on. A container wraps all the different elements of the video into a single file. 

However, just because two video files have the same container, that doesn’t mean their formats are exactly the same! Within the container, there’s the actual video data, audio data and sometimes additional information such as subtitles.

Each of these have their own individual formats. The video and audio streams will each have their own unique formats, referred to as their “codecs”.

The term codec is short for “coder/decoder”. It describes exactly how video or audio is converted from its raw, uncompressed form into something of a more palatable size.

The common video format MP3 is an example of an audio codec. It’s what allows high quality CD audio to be squeezed down to less than a tenth of the original size, without losing much if any subjective quality. Speaking of loss, now’s a good time to explain “lossy” codecs.

“Lossy” Vs “Lossless” Formats

Video contains a ton of data. Analogue film stock, such as movies that have been made on film for the majority of their history, contain an incredible amount of detail. This is why it’s possible to release HD, 4K and 8K remasters of old movies. All you have to do is go back and scan the film frames at a higher resolution. The detail is there, limited only by the resolution of the scanning equipment and the quality of the film grain itself.

For a given image resolution, there’s a ton of information. A single frame of 4K video is equal to a 3840×2160 photo! Compression technology uses various fancy mathematical ways to reduce the amount of information you need to reconstruct an image on screen.

Most of these compression techniques are “lossy”. Which is to say that they throw away some visual information to reduce the size of the video data. However, the loss is usually very minor and well worth the massive reduction in size. Any streaming video, DVD or BluRay content you watch uses lossy compression.

Lossless compression for video is usually only found in the master digital recordings for big-budget film projects or in film archives.

Important Common Codecs

There are hundreds of different codecs and in the past it was an utter nightmare to install all the different codecs you might need to play back video.

Even worse, set-top players usually only supported a small number of codecs, so you would need a computer to convert video into something those machines could understand. These days, almost all video is encoded using one of a small number of codecs.

H.264 – Advanced Video Coding

H.264 is by far the most popular video codec at the time of writing. With just over 90% of all video offered in this common video format. Because H.264 is so popular, most devices (such as smartphones and smart TVs) have special hardware built in to decode H.264 video without putting any strain on the device’s main processor. Which is why even bottom-end smartphones can play HD video without breaking a sweat.

H.265 – The High Efficiency Video Coding

The High Efficiency Video Coding (HEVC) format of video compression has revolutionized video streaming, since it can significantly reduce how much bandwidth you need. It’s designed to be the successor to H.264 and in general uses 25% to 50% less bandwidth to provide the same quality or better quality at the same bandwidth levels.

H.264 is finding great success in the world of streaming, but unlike H.264 there aren’t many devices that have special hardware decoding components for this codec. So while it will save plenty of bandwidth and hard drive space, it will give the target device a real workout. As with H.264, this is likely to change over time, but for now you should keep its limited support in mind before choosing to use it.

MPEG-4

MPEG-4 can get a little complicated. It’s also a very common video codec, but MPEG 4 Part 10 is actually the same as H.264. Early versions of MPEG-4 (e.g. Part 2) use older algorithms that are much less efficient in terms of space for the same level of quality. H.264 has essentially replaced MPEG-4 with a new naming convention.

MP3 – MPEG Audio Layer-3

Just about everyone knows what an MP3 is, since this was the music format that upended the record industry and, eventually, led to the digital streaming music and download model we all know today. What you might not know is that MP3 audio is also pretty common in videos.

Since this format can squeeze CD-quality audio to roughly a tenth of its size without losing too much fidelity, it’s been a mainstay of digital audio for years. Regardless of which video codec a given video container uses, there’s a good chance the audio itself is in MP3 format. Which also has various levels of quality, with the happy medium usually falling around the 128 to 196 Kbps level.

WAV – Waveform Audio Format

The “wave” format has been around for ages and is (in general) an uncompressed digital audio file that represents the original recording waveform precisely. So, as you might expect, it takes up a huge amount of space. At the same quality settings as CD audio, a WAV file should take up roughty as much space as a CD. While it’s not particularly common, a video can contain WAV audio as well.

Common Video Container Formats

The last piece of the puzzle are common container formats. This is what you’ll actually see as the file format of the video. In other words, the file extension you see belongs to the container. Let’s look at the most common ones.

MP4

The MP4 container format is supported by just about every device. It can contain any MPEG-4 format version and H.264. YouTube videos are usually in this common video format.

AVI – Audio Video Interleave

This is one of the oldest video containers and it isn’t used very often anymore, but it’s still widely supported and a lot of existing content is in AVI. The number of codecs that can be used in the AVI container is massive, which is another reason you’d get into cold sweats trying to get an AVI file to play back in the good old wild west days of digital video.

MOV

The MOV container is associated with the Apple QuickTime Player and is its in-house format. Inside a MOV file you are most likely to find MPEG-4 video data. Which is why, in most cases, you can rename a MOV file to an MP4 file and it will work just the same.

The main difference between MOV and MP4 files is that MOV files sometimes have copy protection. This prevents sharing and playing by unauthorized users.

Down The Video Format Rabbit Hole

These common video formats and containers are but the tip of the iceberg. For example, DVDs use MPEG-2, but that’s now rarely used outside of actual DVD discs you’d buy at a store. There are also professional video formats (e.g. ProRes RAW) and common formats traded on the internet (e.g. MKV).

It would require, literally, an entire book to cover them all. However, the world is standardizing towards H.264 and H.265. So if you make videos at any point, either of those are likely to be a safe bet. With H.264 currently the safest bet of them all!