Video codec: not as scary as it sounds



Getting on first terms with codecs and video formats
What and what for
The first obvious question: why do we need any special encoding at all? I mean, if we were using a digital camera, we already have some kind of files. So why not use those directly? The thing is, the so called raw format in professional cameras turns out to be a sequence of separate pictures, each one stored in highest quality. And now multiply this by a number of frames per second (where, by the way, the standard is steadily drifting from old school cinematic 24 fps to trendy 30 fps).

So to store an one minute long 920x1080 video in RAW you'll need up to 20 gigabytes (sic!) of disc space. And don't forget that good old Full HD also slowly but surely turns from an extravagance into a baseline, giving way to new premium resolutions of 2.5K and even 4K. All right, now compare the volume of data we're talking about with the capabilities of an average Internet connection. As you can see, if this was the case, the movies would come to store shelves exclusively on multi-terabyte hard drives while the technology of streaming would not even be available for mere mortals.

Thus codecs are absolutely needed algorithms for optimizing the size of media content while maintaining an acceptable, sometimes even unnoticeable with a bare eye level of quality loss. Let's look at an example of logic for such encoding. In RAW, every pixel of every frame always stands as a sequence of ones and zeros of equal length. But what if, say, half of this frame is occupied by an evenly lit white background? Or the opposite, a dark tunnel where there's nothing to be seen? Or simpler still, you have objects with low reflectivity just standing there, which means nothing changes on their surfaces during the whole scene? Instead of uploading a huge amount of unnecessary data, it's sufficient to feed our algorithm with coordinates of an area that should be covered with one plain color, completed with the time stamps. Voila, we've just shrank our data several times, using just the most primitive of encoding techniques. Then applying the aforementioned H.264 codec to the same one minute long video will compress it to 600 Mb, which is 30 times less!!! The arsenal and effectiveness of such techniques is exactly what defines the difference between codecs and as a result – their popularity.
The choice
H.264 — the most widely used codec for video streaming today. It combines a good rate of data compression with compatibility to almost any device. Initially it had a range of various profiles for devices of different power, but today most modern processors are more than capable of working with the high quality one.

VP9
– arguably one of the strongest contenders for a leading codec in next few years. Developed by Google under an open source model, it boasts a higher performance than H.264. That being said, it still doesn't have a similar level of support on different platforms.

AAC – the most popular audio codec, developed back in the late 90's, it combines decent quality with relatively modest bitrates as well as compatibility with pretty much any device.

MP3 – a folk hero of audio encoding for decades, it is still widely used today mostly for sharing musical tracks and audio books. However, it never gained the same level of popularity in video streaming circles.
Not-a-Format
Speaking of video codecs, it's important to also highlight a very typical mistake: confusing codecs with video formats. A "format" is that very feature responsible for a three-letter extension after a period in every filename, be it .avi, .mov or any other. To understand it's role you can remember that media formats are also frequently called containers or wrappers. They serve for putting the whole package of media experience together, including visuals, multiple sound tracks (e.g., dubs in different languages), subtitles, timecodes and the rest of the metadata. At the same time, the codecs inside can potentially be of any type, at least as far as compatibility allows.
Current choice and future
As for today, we would recommend using H.264 as your main video encoding option for online streaming, complete with good old AAC for audio. The main reason is simple: maximum compatibility with all types of hardware and software which guarantees maximum coverage. That being said, we strongly suggest you take a close look at the digital landscape at least once a year – a new more effective codec might burst out onto the scene and gain popularity as quickly as in a few months. And when it happens, whatever your projects are, as long as they're connected to video in some way you'll only benefit from riding the wave of that trend!
Artem
Chief Technology Officer, Co-Founder, IT Manager with vast experience in software development, managed IT services, with stress on video-processing and AdTech