Compression Formats for Video and Graphics

Compression Formats

There are many types of compression used in graphics and video, with lossy and lossless results. In this section, we'll list those types that are commonly used within the digital realm of the computer: whether used as an intermediate format for video editing purposes, creating outputs for the web, DVD's and other applications. We'll also explore those compression formats used in various cameras and recorders. In later sections, we'll look at compression in the world of videotape formats and higher end file formats used in the creation of digital intermediates (DI) to print out to film from digital data. Some of these topics overlap as we use similar compression techniques for the creation of digital videotape as well as digital video files.

Graphic File Formats

We'll start off with graphic formats, as the first in the list, JPEG, a still image format, is used as the basis for several different codecs used in video compression. The other formats listed here are commonly used in print and video applications for broadcast and the web, either as single independent images incorporated into a video program, or as "image sequences" which is basically a group of images that when played back one after the other, produce video animation.

JPEG: (Joint Photographic Experts Group). Standard first issued in 1992. Compression method for still images. Usually lossy compression. The compression is scalable, and the higher the compression (ratio), the more artifacts become noticeable. 8-bit per channel color depth. One of the main benefits of the format is that when compressed, it creates relatively small files. Because of this, it is an easy format for interchange across the internet. One of the downsides of the format is that if an image is opened (decompressed), then re-saved, it is re-compressed, leading to further quality loss in the form of visual artifacts.

TIFF: (Tagged Image File Format) Copyright is currently owned by Adobe Systems. Format is available on Mac and Windows systems. Supports 16-bit depth per channel as well as support for 32-bit float space. TIFF supports several compression schemes, lossy and lossless. One option is called LZW compression, which produces lossless compression while reducing file sizes. TIFF images include "tags", which is a way of storing various information about the graphic. For example, the various available color spaces and compression options are stored as tags. Another flexibility of TIFF is support for an alpha channel (transparency information). As well as a file format used to create graphics for print and video applications, its also been used in the past to create TIFF image sequences for use in high-end environments like animation or 2k captures of film for digital intermediates, which was suitable due to the lossless, high quality nature of uncompressed TIFF's. Currently, most high-end digital film work (digital intermediate) is processed with DPX files, discussed in Beyond HD.

TGA: Often referred to as Targa. Originally developed by Truevision. File format used mostly in animation (image sequences) and standard definition video. Format supports an alpha channel to preserve transparency information and lossless compression. Uses 8-bit per channel color depth.

PSD: Adobe's Photoshop supports many different file types and compression schemes. The PSD format, the native format to Photoshop, supports multiple layers (think of a layer like its own individual graphic image), alpha channels, multiple color spaces and bit depths. Color spaces available include RGB, CMYK, Grayscale, Bitmap, and Lab color. The support for multiple layers and an alpha channel makes it ideal for compositing multiple images together, creating templates for DVD menus, and many other print and video applications. Graphics created in Photoshop can be saved out into many other file formats including TIFF, PICT, BMP, TGA, JPEG and others. PSD file sizes can run somewhat large due to the uncompressed nature of the format. Supports multiple bit depths up to 32-bit per channel. Available on Mac and Windows platforms.

Bitmap: (BMP) Graphic format found primarily in Windows. Usually uncompressed, so file sizes are relatively large.

Video Editing/Video Acquisition/DVD/Web Formats

The formats listed below include video editing formats, DVD formats and web distribution formats. Some of the following formats are meant for video editing (and mastering), where other formats are intermediate editing formats or "delivery" formats and are generally fairly compressed for that purpose. Some of the following formats are the compression format used for acquisition as well.

First off, let's start with a couple terms that do not relate to a particular compression format, but relates to compression in general. A longer definition for the term codec is found in the section Compression Fundamentals.

Codec: Stands for compress/decompress, or encode/decode. Also called compressor. The specific mathematical algorithm for a given format used to compress and decompress video for storage, display, transmission, editing, manipulation, etc. Can be hardware or software based.

DCT: (Discrete Cosign Transform). Most common flavor is DCT Type II. DCT is not a particular video format or codec, but it’s a compression scheme for how most video is (spatially) compressed, including videotape formats, covered in the next sections. Method reduces spatial detail within the frame. Notably, part of the process involves transforming each block of 8x8 pixels into a "macroblock", which basically processes those groups of pixels together, and somewhat separate from other pixels. There are different compression schemes that use different size macroblocks as well. Common formats using DCT processing include JPEG, DV, MJPEG and MPEG. This compression scheme is used for videotape and software compression formats. The new RED camera, a 4k digital acquisition camera uses wavelet style compression to avoid the "blocky" constraints of DCT macroblock compression.

Apple's Animation: (Editing format) A lossless codec created by Apple often used for high-end compositing applications, primarily used as an intermediate step for moving between different applications while maintaining full quality. The format is uncompressed (using RGB color space) so the full range of data is maintained, leading to high data rates and needing large amounts of hard drive storage space (if used at 100% quality). It is a lossy codec if used at quality settings below 100%. Often used to preserve an "alpha channel". In other words, image transparency is able to be retained using this codec when moving between various applications. In Apple's QuickTime, to preserve an alpha channel, the user would not only choose the Animation codec, but would also choose a setting labeled as Millions of Colors+. The "+" indicates preserving the alpha channel. The Animation codec is available in a wide range of Mac applications including After Effects, Final Cut Pro and Motion.

Apple's Uncompressed: (Editing format) Apple developed these codecs for use in Final Cut Pro and other video applications. There are 8-bit and 10-bit versions, with color sampling at 4:2:2. Both codecs default to working in YCbCr coding, but will also work in RGB. Capture cards by AJA and Blackmagic Design can use Apple's codecs for capture and sequence presets. These codecs are not designed for capturing "native" formats like DV through FireWire. Although if the DV is captured through a capture card, then these codecs can be used to capture and work with the video uncompressed while it resides in the computer. The uncompressed codecs only exist in the computer. Once video is output to tape, the compression scheme of that video deck is used. The advantage of working uncompressed in the computer is to avoid artifacts that build up when creating effects in compressed editing environments and for mastering at the highest possible quality. This is especially important for image stressing applications like chroma keying. Typically, one would capture video first at a lower resolution for offline editing, then re-capture with the uncompressed codec for high quality mastering.

Apple's ProRes 422: (Editing format) A new codec developed by Apple for editing SD and HD video at a fraction of uncompressed file sizes and data rates. This codec comes available with the release of Final Cut Studio 2 (Final Cut 6). Apple touts this format as maintaining extremely high quality, "indistinguishable from the original", so it can be used for offline editing or for high quality online mastering. In HD, it supports the full, square pixel 16x9 frame sizes (1920x1080 and 1280x720). In addition, it uses 4:2:2 color sampling (the same professional standard used in most high-end SD and HD video) and 10-bit depth, which is needed to preserve subtle shifts in levels (like gradients) without artifacts known as banding or contouring. The format uses VBR (variable bit rate encoding) to provide efficient compression and to allow for higher data rates for complex detail. There is support for 2 different target bit rates: ProRes: 145 Mbit/sec and ProRes HQ (high quality): 220 Mbit/sec. These data rates are lower than uncompressed standard definition video. It uses spatial compression and no temporal compression (referred to as I-frame video). Another main benefit to ProRes 422: optimized support for multiple streams of real time effects.

Avid DNxHD: (Editing format) Developed prior to Apple's ProRes 422. For use in Avid non-linear editing and based on the MXF open-format standard which means that the data can be interchanged with other systems that support MXF. Codec is available for Mac and Windows platforms, and can even be "wrapped" in QuickTime for QuickTime applications like After Effects. Provides excellent HD quality at a fraction of uncompressed HD video, even after multiple generation re-encoding. Supports several streams of real time effects, multicamera editing and collaborative sharing across Avid Unity network systems. Support for full, square pixel HD frame sizes (1920x1080 and 1280x720), 4:2:2 color sampling, and 8-bit or 10-bit depth. Supports different bit rates, 145 Mbit/sec and 220 Mbit/sec for different quality levels. Scalable so that future refinement of data rates and resolutions is possible. Both Apple's ProRes 422 and Avid's DNxHD are mastering quality formats, and have similar PSNR (peak signal to noise ratio) values, which is a measure of the fidelity of the encoded video.

MPEG: Motion Picture Experts Group. Includes a broad range of standards, some of which are listed next.

MPEG-1: (This format is used for delivery purposes). Compression standards used for video CD's, internet distribution. For video and audio compression. For video, it is limited to progressive images. Codec is not as efficient as newer standards, and so the quality is not high at the same bit rates of newer formats. One version still in heavy use is for audio: MP3 is the short name for MPEG-1 Audio Layer 3. Although MPEG-1 video is outdated by newer compression technology, it is still highly compatible on many computers, which is more than can be said for some of the newer codecs being used, which may or may not be available on a particular computer.

MPEG-2: (This format is used mostly for delivery purposes). A temporal compression standard used in many applications including: standard definition DVD's, and digital cable transmission (SD and HD). Its also used as an acquisition format for all HDV video recording, and Sony's XDCAM HD and EX cameras/decks. May use variable or constant bit rate encoding. The chroma sampling is 4:2:0 for the applications described above, but there are profiles that support other chroma sampling schemes. DVD players will usually up-sample this to 4:2:2 on output for display.

Encodes using three kinds of frames for temporal compression in a structure known as a GOP (group of pictures):

• I-picture (Intra): Reference to an individual compressed frame with no relation to other pictures. Has spatial but no temporal compression.
• P-picture (Predictive): Uses the previous I or P picture for motion compensation and reference for prediction.
• B-picture (Bi-directionally predictive): Uses the previous and next I or P pictures for motion compensation.

Optimized at high bit rates, but not as efficient at lower bit rates. Subsequently, formats like HDV which use a low 25Mbit/sec bit rate may suffer from blocky motion artifacts.

MPEG-4: (Mostly used for delivery purposes). Used for a wide range multimedia applications. Multiple profiles for various applications. Used for everything from web compression, video conferencing to high end video tape recording (HDCAM SR uses the MPEG-4 Studio Profile). As end users, most will utilize MPEG-4 for delivery purposes, creating lossy compression.

H.263: (Delivery format) Codec works best at low data rates. Medium quality. Because of heavy temporal compression, it is not ideal for material with a lot of motion. Commonly found in videoconferencing. A newer standard, H.264, is better optimized for higher quality at lower (and higher) data rates and is more efficient than H.263.

H.264: (Acquisition and Delivery format) Technically the same as MPEG-4 Part 10, also known as AVC (Advanced Video Coding). Standardized in 2003. H.264 is increasingly being used in multiple applications. There are several profiles for various applications. Highly scalable compression produces great quality results at lower data rates than in previous standards (such as MPEG-2) and it is a very efficient codec. H.264 can be configured as I-frame only (spatial compression only) or as part of a Long-GOP structure using temporal compression similar to MPEG-2. Some applications that H.264 is being used for:

• Blu-ray DVD format has adopted this format.
• Satellite Broadcast including DirecTV (currently for HDTV).
• Web compression.
• iPod and iPhone video.
• Apple TV.
• Videoconferencing.
• Video acquisition for certain HD cameras.

To get an idea of the incredible quality of H.264 (and at lower data rates than ever before), check out http://www.apple.com/quicktime/guide/hd/ to download HD content in 720p or 1080p from Apple's website. Although the files will require a high bandwidth internet connection, the files are still much smaller than the same quality with other compression methods.

Note: H.264 codec must be present to play back H.264 material on a computer. For example, H.264 became available to Apple computers with the release of QuickTime 7. But those running QuickTime versions 6 and earlier will not be able to play back the video.

AVC-Intra: (Acquisition format) Although it goes by its own name, this is just one variation of H.264 as implemented by Panasonic. It uses intra-frame (spatial) compression, so there is no temporal compression artifacts. It produces high quality images at very low bit rates. This format is one of two compression formats available on some Panasonic HD cameras using P2 cards for video recording (the other format being DVCPRO HD). There are substantial benefits to recording in AVC-Intra over DVCPRO HD.

As an example, both formats can record at 100Mbit/sec, yet the AVC-Intra codec can record higher quality for the same storage and bit rate.

• AVC-Intra- 1920x1080, 10-bit, 4:2:2
• DVCPRO HD- 1280x1080, 8-bit, 4:2:2

AVC-Intra has two recording modes: 50Mbit/sec and 100Mbit/sec. The 50Mbit/sec version uses anamorphic HD frame sizes, and 4:2:0 color sampling, where the 100Mbit/sec version uses full size HD frame sizes and 4:2:2 color sampling. Panasonic claims the quality of the 100Mbit/sec version is comparable to recording to a HD-D5 tape deck.

AVCHD: (Acquisition format) This is a consumer version of H.264 as implemented by a number of manufacturers for consumer HD camera recording. This version of H.264 uses temporal compression (inter-frame) in a similar Long-GOP structure as MPEG-2. This format uses 4:2:0 color sampling and 8-bit depth. This format can be recorded to a number of different mediums- DVDs, SD cards and HDDs.

Sorenson Video 3: (Delivery format) Compression scheme used for internet distribution and developed by Sorenson Media. Scalable codec can produce varying degrees of quality (at varying levels of bandwidth). Uses temporal and spatial compression. Used in Apple's QuickTime and Adobe's Flash.

Photo JPEG: (Editing format) Based on the JPEG compression algorithm. Used in video editing at various quality levels. It is CPU intensive at high quality and can produce stunning results, but the bandwidth costs are high and other codecs could create similar quality at lower data rates and file sizes. Not as high color fidelity as MJPEG when used at lossy settings. Color is down-sampled to 4:2:0.

MJPEG: (Motion JPEG) (Editing format). Not to be confused with MPEG. There are two MJPEG codecs used in QuickTime: MJPEG-A and MJPEG-B. Also based on the JPEG compression algorithm. These formats can be said to create better results than Photo JPEG as the compression with MJPEG is field based, leading to better color fidelity with interlaced images. Color is sampled at 4:2:2. Similar to Photo JPEG, if quality is at 100% the codecs produce almost lossless results. These formats are also optimized for certain hardware decoders (capture cards). MJPEG is also found in Windows "wrapped" AVI's (Audio Video Interleaved).

Native Video Editing Formats

There are several videotape compression formats (covered in the next section) that are natively supported in the Mac via QuickTime. Check out the documentation for your editing system to see if all of the following formats are native to that system. All of the following are native in Final Cut Pro (starting with version 5.1.2). As we discussed in Native Editing, there are certain formats like DV that are captured into the computer in the same format (same compression scheme) as the video is on tape. We'll cover the details of these tape formats more in the sections SD Tape Formats and HD Tape Formats. We mention these formats here because they do exist in the computer completely separately from their videotape counterparts. These formats have low data rates and small file sizes so they are easy to edit with, and have good quality to boot. You can transcode video from tape into one of these formats in the computer: for example, capture high resolution HD video into DVCPRO HD, a fairly compressed format. You can simply edit with these compression formats in an offline edit and re-capture uncompressed for online editing, or if your budget doesn't allow this, you can finish and master with these formats. You can even output these formats to a higher-end videotape format- getting a little of the best of both worlds. The only downside is that highly compressed formats are not as good for heavy effects done in post production, like chroma keying and will lose quality over multiple generations of re-encoding.

A few native formats used for offline and online editing:

DV: This encompasses MiniDV, DVCAM, DVCPRO (25). Standard definition video, uses spatial compression. These various formats are the same file format in the computer, although each is handled slightly differently on tape. Often used to down-convert high-end SD and HD video for offlining purposes. Used often for online editing as well, but is not ideal for heavy effects work like chroma-keying. Uses 5:1 spatial compression, 4:1:1 color sampling, and 8-bit depth.

DVCPRO 50: Panasonic format, standard definition. Less compressed and better color space than DV. Offline and online editing format. Uses 4:2:2 color sampling and 8-bit depth.

DVCPRO HD: HD format by Panasonic. Fairly compressed (approx 7:1), but good color space. This format is heavily used in offline and/or online editing. Uses 4:2:2 color sampling and 8-bit depth.

HDV: Format uses MPEG-2 Long-GOP recording at a 25Mbit/sec bit rate. Although HDV is increasingly being found as a native format on different editing systems, the format is not ideal for offline or online editing. It is highly compressed, both spatially and temporally, and has low color resolution. It uses 4:2:0 color sampling and 8-bit depth. The main reason it is not great for offline editing is because of the intense computing needed to edit MPEG-2 in real time. In other words, the computer can get bogged down easily with effects, re-conforming and other tasks. If you shot in HDV, you may want to transcode to another format for editing for more convenience and less computing overhead, but its not necessary if its native to that editing system.