Information Technology

Codecs and Wrappers for Digital Video

In the last Greatbear article we quoted sage advice from the International Association of Audiovisual Archivists: ‘Optimal preservation measures are always a compromise between many, often conflicting parameters.’ [1]

While this statement is true in general for many different multi-format collections, the issue of compromise and conflicting parameters becomes especially apparent with the preservation of digitized and born-digital video. The reasons for this are complex, and we shall outline why below.

Lack of standards (or are there too many formats?)

Carl Fleischhauer writes, reflecting on the Federal Agencies Digitization Guidelines Initiative (FADGI) research exploring Digital File Formats for Videotape Reformatting (2014), ‘practices and technology for video reformatting are still emergent, and there are many schools of thought. Beyond the variation in practice, an archive’s choice may also depend on the types of video they wish to reformat.’ [2]

We have written in depth on this blog about the labour intensity of digital information management in relation to reformatting and migration processes (which are of course Great Bear’s bread and butter). We have also discussed how the lack of settled standards tends to make preservation decisions radically provisional.

In contrast, we have written about default standards that have emerged over time through common use and wide adoption, highlighting how parsimonious, non-interventionist approaches may be more practical in the long term.

The problem for those charged with preserving video (as opposed to digital audio or images) is that ‘video, however, is not only relatively more complex but also offers more opportunities for mixing and matching. The various uncompressed-video bitstream encodings, for example, may be wrapped in AVI, QuickTime, Matroska, and MXF.’ [3]

What then, is this ‘mixing and matching’ all about?

It refers to all the possible combinations of bitsteam encodings (‘codecs’) and ‘wrappers’ that are available as target formats for digital video files. Want to mix your JPEG2000 – Lossless with your MXF, or ffv1 with your AVI? Well, go ahead!

What then is the difference between a codec and wrapper?.

As the FADGI report states: ‘Wrappers are distinct from encodings and typically play a different role in a preservation context.’ [4]

The wrapper or ‘file envelope’ stores key information about the technical life or structural properties of the digital object. Such information is essential for long term preservation because it helps to identify, contextualize and outline the significant properties of the digital object.

Information stored in wrappers can include:

  • Content (number of video streams, length of frames),
  • Context (title of object, who created it, description of contents, re-formatting history),
  • Video rendering (Width, Height and Bit-depth, Colour Model within a given Colour Space, Pixel Aspect Ratio, Frame Rate and Compression Type, Compression Ratio and Codec),
  • Audio Rendering – Bit depth and Sample Rate, Bit Rate and compression codec, type of uncompressed sampling.
  • Structure – relationship between audio, video and metadata content. (adapted from the Jisc infokit on High Level Digitisation for Audiovisual Resources)

Codecs, on the other hand, define the parameters of the captured video signal. They are a ‘set of rules which defines how the data is encoded and packaged,’ [5] encompassing Width, Height and Bit-depth, Colour Model within a given Colour Space, Pixel Aspect Ratio and Frame Rate; the bit depth and sample rate and bit rate of the audio.

Although the wrapper is distinct from the encoded file, the encoded file cannot be read without its wrapper. The digital video file, then, comprises of wrapper and at least one codec, often two, to account for audio and images, as this illustration from AV Preserve makes clear.

Codecs and Wrappers

Diagram taken from AV Preserve’s A Primer on Codecs for Moving Image and Sound Archives

Pick and mix complexity

Why then, are there so many possible combinations of wrappers and codecs for video files, and why has a settled standard not been agreed upon?

Fleischhauer at The Signal does an excellent job outlining the different preferences within practitioner communities, in particular relating to the adoption of ‘open’ and commercial/ proprietary formats.

Compellingly, he articulates a geopolitical divergence between these two camps, with those based in the US allegedly opting for commercial formats, and those in Europe opting for ‘open.’ This observation is all the more surprising because of the advice in FADGI’s Creating and Archiving Born Digital Video: ‘choose formats that are open and non-proprietary. Non-proprietary formats are less likely to change dramatically without user input, be pulled from the marketplace or have patent or licensing restrictions.’ [6]

One answer to the question: why so many different formats can be explained by different approaches to information management in this information-driven economy. The combination of competition and innovation results in a proliferation of open source and their proprietary doubles (or triplets, quadruples, etc) that are constantly evolving in response to market ‘demand’.

Impact of the Broadcast Industry

An important area to highlight driving change in this area is the role of the broadcast industry.

Format selections in this sector have a profound impact on the creation of digital video files that will later become digital archive objects.

In the world of video, Kummer et al explain in an article in the IASA journal, ‘a codec’s suitability for use in production often dictates the chosen archive format, especially for public broadcasting companies who, by their very nature, focus on the level of productivity of the archive.’ [7] Broadcast production companies create content that needs to be able to be retrieved, often in targeted segments, with ease and accuracy. They approach the creation of digital video objects differently to how an archivist would, who would be concerned with maintaining file integrity rather ensuring the source material’s productivity.

Furthermore, production contexts in the broadcast world have a very short life span: ‘a sustainable archiving decision will have to made again in ten years’ time, since the life cycle of a production system tends to be between 3 and 5 years, and the production formats prevalent at that time may well be different to those in use now.’ [8]

Take, for example, H.264/ AVC ‘by far the most ubiquitous video coding standard to date. It will remain so probably until 2015 when volume production and infrastructure changes enable a major shift to H.265/ HEVC […] H.264/ AVC has played a key role in enabling internet video, mobile services, OTT services, IPTV and HDTV. H.264/ AVC is a mandatory format for Blu-ray players and is used by most internet streaming sites including Vimeo, youtube and iTunes. It is also used in Adobe Flash Player and Microsoft Silverlight and it has also been adopted for HDTV cable, satellite, and terrestrial broadcasting,’ writes David Bull in his book Communicating Pictures.

HEVC, which is ‘poised to make a major impact on the video industry […] offers to the potential for up to 50% compression efficiency improvement over AVC.’ Furthermore, HEVC has a ‘specific focus on bit rate reduction for increased video resolutions and on support for parallel processing as well as loss resilience and ease if integration with appropriate transport mechanisms.’ [9]

CODEC Quality Chart3Increased compression

The development of codecs for use in the broadcast industry deploy increasingly sophisticated compression that reduce bit rate but retain image quality. As AV Preserve explain in their codec primer paper, ‘we can think of compression as a second encoding process, taking coded information and transferring or constraining it to a different, generally more efficient code.’ [10]

The explosion of mobile, video data in the current media moment is one of the main reasons why sophisticated compression codecs are being developed. This should not pose any particular problems for the audiovisual archivist per se—if a file is ‘born’ with high degrees of compression the authenticity of the file should not ideally, be compromised in subsequent migrations.

Nevertheless, the influence of the broadcast industry tells us a lot about the types of files that will be entering the archive in the next 10-20 years. On a perceptual level, we might note an endearing irony: the rise of super HD and ultra HD goes hand in hand with increased compression applied to the captured signal. While compression cannot, necessarily, be understood as a simple ‘taking away’ of data, its increased use in ubiquitous media environments underlines how the perception of high definition is engineered in very specific ways, and this engineering does not automatically correlate with capturing more, or better quality, data.

Like error correction that we have discussed elsewhere on the blog, it is often the anticipation of malfunction that is factored into the design of digital media objects. These, in turn, create the impression of smooth, continuous playback—despite the chaos operating under the surface. The greater clarity of the visual image, the more the signal has been squeezed and manipulated so that it can be transmitted with speed and accuracy. [11]

MXF

Staying with the broadcast world, we will finish this article by focussing on the MXF wrapper that was ‘specifically designed to aid interoperability and interchange between different vendor systems, especially within the media and entertainment production communities. [MXF] allows different variations of files to be created for specific production environments and can act as a wrapper for metadata & other types of associated data including complex timecode, closed captions and multiple audio tracks.’ [12]

The Presto Centre’s latest TechWatch report (December 2014) asserts ‘it is very rare to meet a workflow provider that isn’t committed to using MXF,’ making it ‘the exchange format of choice.’ [13]MXF

We can see such adoption in action with the Digital Production Partnership’s AS-11 standard, which came into operation October 2014 to streamline digital file-based workflows in the UK broadcast industry.

While the FADGI reports highlights the instability of archival practices for video, the Presto Centre argue that practices are ‘currently in a state of evolution rather than revolution, and that changes are arriving step-by-step rather than with new technologies.’

They also highlight the key role of the broadcast industry as future archival ‘content producers,’ and the necessity of developing technical processes that can be complimentary for both sectors: ‘we need to look towards a world where archiving is more closely coupled to the content production process, rather than being a post-process, and this is something that is not yet being considered.’ [14]

The world of archiving and reformatting digital video is undoubtedly complex. As the quote used at the beginning of the article states, any decision can only ever be a compromise that takes into account organizational capacities and available resources.

What is positive is the amount of research openly available that can empower people with the basics, or help them to delve into the technical depths of codecs and wrappers if so desired. We hope this article will give you access to many of the interesting resources available and some key issues.

As ever, if you have a video digitization project you need to discuss, contact us—we are happy to help!

References:

[1] IASA Technical Committee (2014) Handling and Storage of Audio and Video Carriers, 6. 

[2] Carl Fleischhauer, ‘Comparing Formats for Video Digitization.’ http://blogs.loc.gov/digitalpreservation/2014/12/comparing-formats-for-video-digitization/.

[3] Federal Agencies Digital Guidelines Initiative (FADGI), Digital File Formats for Videotape Reformatting Part 5. Narrative and Summary Tables. http://www.digitizationguidelines.gov/guidelines/FADGI_VideoReFormatCompare_pt5_20141202.pdf, 4.

[4] FADGI, Digital File Formats for Videotape, 4.

[5] AV Preserve (2010) A Primer on Codecs for Moving Image and Sound Archives & 10 Recommendations for Codec Selection and Managementwww.avpreserve.com/wp-content/…/04/AVPS_Codec_Primer.pdf, 1.

‎[6] FADGI (2014) Creating and Archiving Born Digital Video Part III. High Level Recommended Practices, http://www.digitizationguidelines.gov/guidelines/FADGI_BDV_p3_20141202.pdf, 24.
[7] Jean-Christophe Kummer, Peter Kuhnle and Sebastian Gabler (2015) ‘Broadcast Archives: Between Productivity and Preservation’, IASA Journal, vol 44, 35.

[8] Kummer et al, ‘Broadcast Archives: Between Productivity and Preservation,’ 38.

[9] David Bull (2014) Communicating Pictures, Academic Press, 435-437.

[10] Av Preserve, A Primer on Codecs for Moving Image and Sound Archives, 2.

[11] For more reflections on compression, check out this fascinating talk from software theorist Alexander Galloway. The more practically bent can download and play with VISTRA, a video compression demonstrator developed at the University of Bristol ‘which provides an interactive overview of the some of the key principles of image and video compression.

[12] ‘FADGI, Digital File Formats for Videotape, 11.

[13] Presto Centre, AV Digitisation and Digital Preservation TechWatch Report #3, https://www.prestocentre.org/, 9.

[14] Presto Centre, AV Digitisation and Digital Preservation TechWatch Report #3, 10-11.

Posted by debra in Information Technology, Video Tape, 1 comment

Audio data recovery from external USB drive using ddrescue

High resolution audio and video digital tape conversions can use large amounts of computer storage. 8 bit uncompressed Standard Definition (SD) PAL video runs at 70 GB per hour and 24 bit 96 kHz audio files at 2 GB per hour.

As a result of this many of our analogue to digital tape transfers require the use of external storage, usually USB 2.0 portable hard drives, to supply the copied digital transfers back to the customer. Some drives supplied by customers have not been of great quality and not designed to be sent about in the post. One such drive we had recently, a Sony Vaio branded 2.5″ USB drive wouldn’t copy certain directories of important files with the Mac OS Finder or the Windows Explorer. While most of the drive copied this certain folder always resulted in a crashed computer!

Thanks to GNU/Linux we have a bit more power and information at our disposal about hard drives and IDE or USB interfaces. It’s always best practice to copy as much information from the drive or mirror it before attempting any other types of data recovery or file system repair. Using the standard dd

Posted by greatbear in Audio Tape, GNU / Linux, Information Technology, Video Tape, 0 comments

HTC Desire in the toilet

HTC Desire android mobile smart phone

Not our normal type of post but the new HTC Desire Android smart phone has been so useful with it’s open source applications, fast speed and wifi that I couldn’t believe I’d just put the phone in my shirt pocket and leant over the toilet!

Panicking, I fished it out and messed about trying to shut it down, when I should have just whipped the back off and the battery out. I did manage to get the battery out pretty quickly but not before I’d seen the screen start to make weird colours and odd lines. The thought of buying a new phone at full price was too much so I pulled out a handy bottle of Isopropyl alcohol (IPA) we use for cleaning tape and video record and repro heads. This is 99% pure IPA, not cheaper ‘rubbing alcohol’ that is often 70% IPA with 30% water. You really don’t want to make your phone any worse!

When I’d calmed down a bit I checked the moisture pads that show if the phone’s had water contact. These had gone pink confirming the toilet dunking so no warranty repairs here! Having little to lose I removed the 4 small torx head screws and the two even smaller crossheads screws under the rear cover. Two of the Torx screws have ‘void’ stickers over them so again once you undo them you’ve lost your warranty!

Now the bottom cover over the USB connecter can come up as can the inner cover which will reveal the wifi / bluetooth aerials. I didn’t remove the screen but ‘carefully’ poured IPA all over the bits that smelt of wee!

99% pure IPA will evaporate very quickly but any residual moisture should go before you can think about putting the battery and other cards back in. Many online sources recommend uncooked rice to absorb moisture but purpose made desiccants are best. I used several silica gel sachets in a freezer bag with the disassembled phone and left this overnight.

Thankfully this all worked, the phone starts, the screen works it’s alive!

The key points when you have this kind of phone disaster are:

  1. Get the phone out of the water / wee
  2. Get the battery out as soon as possible
  3. Remove all the SIM / memory cards
  4. Shake or carefully pat dry as much liquid as you can
  5. Any liquid contaminant that isn’t pure water and contains sugars or salts must be cleaned off
  6. If you try IPA make sure it’s high purity 99% is best

Finally, we’re tape transfer specialists not phone repair specialists so I offer no guarantee with all this but it did work for me.

Posted by greatbear in Information Technology, 1 comment

Focusrite Saffire LE SPDIF problems

Focusrite Saffire LE

Focusrite Saffire LE

I’ve owned one of these Saffire LE firewire interfaces for a few years now and it’s very useful as a backup to my RME interfaces especially when I need to record and transfer audio at another location as it’s small, light and portable. It’s a very good sounding interface and  it even is reported to work under Linux which I’ve tried but unfortunately not with Audacity as this that doesn’t use the Jack server.

I’ve had an ongoing issue with the SPDIF inputs though. Under Windows XP and MacOS X it refuses to record more than a few mninutes if locked to another digital unit such as DAT machine and the Saffire software reports the firewire interface has lost it’s connection. This has rendered all SPDIF transfers impossible to do correctly.

The support at Focusrite is very good, you can actually speak to someone! We tried using an external power supply unit (PSU), different firewire cables, checking the type of firewire interface in the Windows machine and upgrading the firmware under MacOS X but all to no avail. This is a repeatable issue under several different computers and OS’s.

Posted by greatbear in Audio Tape, Information Technology, 0 comments

Acer 1360 laptop motherboard replacement

I recently had a dead Acer 1360 laptop on my hands.

The power supply tested fine so it seemed likely the main board was at fault. This is a one piece board and of course it requires a full dismantling of the laptop to remove and replace.

I took a chance on an eBay board that arrived badly packaged and wrapped in bubblewrap not anti-static packaging. It stank of nicotine too!

acer_1360_motherboard

Acer 1360 bare motherboard

Disassembly was quite easy. Only three types of screws, big and small black and silver, very idiot proof. I didn’t have a service manual or takeapart guide but this is what I did, no pictures I’m afraid:

  1. Connected my ESD wrist strap.
  2. Remove battery, hard drive, ram and wireless cards (easy!)
  3. Remove the four small silver screws in the battery compartment.
  4. Gently prise off the small covers around each screen hinge.
  5. Remove more silver screws and remove button / fan plate, keyboard and top cover (can’t remember the order but it’s quite obvious)
  6. Gently prise up ribbon cable connectors and remove keyboard and trackpad cables.
  7. Time to remove the screen – The power and video connectors on the main board need to be removed, this needs very small fingers but I used a small ceramic screwdriver to help prise it up.
  8. Remove the two black screws each side holding the screen to the main laptop body.
  9. Put screen somewhere safe while feeding the two wireless antenna cables out.
  10. Start removing everything else! All the long black screws from underneath and all the small silver screws inside the laptop some holding metal shielding.
  11. Hopefully now you should be down to the bare motherboard – remove it and put your new one in!
  12. I hope you remembered the order all the bits went back on? It’s got to be right and there’s a few bits that have to be done in the right order on assembly.
  13. I found I had to remove some parts from the old motherboard that were missing on the replacement one.
  14. Have you got ALL the screws?

I found that my new motherboard worked perfectly first time all apart from the Trackpad and buttons! We just used an external mouse instead!

acer_1360_damage_detail

Damaged inductor

I did find out later that the original damage had been caused by a heavy fist slamming into the trackpad in frustration with Windows. Inside the case there were some plastic parts that had broken off one of these inductors. This would also likely explain the broken trackpad……

Posted by greatbear in Information Technology, 6 comments

PC World terrible customer service and support

altered pc world logo

A customer recently came to me after an unhelpful, incorrect and ultimately expensive experience at PC World.

Their Dell Inspiron laptop had stopped working, after initially refusing to look at it, the ‘tech’ person grudgingly agreed after a complaint to his manager. After being told it was thought the power supply had stopped working, he took it away and ultimately concluded the motherboard needed replacing.

Of course it didn’t and just needed another power supply which was less than £20.00 for a recycled one on eBay!

Unfortunately the customer had already bought another laptop on this expert’s advice which she didn’t need. Please don’t go to PC World for advice!

Posted by greatbear in Information Technology, 0 comments

Advent 7055 laptop overheating repair

Advent 7055 running Ubuntu 8.04Just recently I’d been asked to repair what seems to be a common problem with PC type laptops over a couple of years old. Dust gets trapped in the copper fins and the fan of the CPU heatsink and reduces the efficiency of the cooling resulting in regular power off’s by the laptop as it gets over it’s upper temperature setting.

The fix is relatively easy, remove the heatsink unit, blow it clean with compressed air, clean the old CPU heatsink paste off with isopropyl alcohol (IPA) and spread a thin but even film of good quality heatsink paste on the CPU. Put it all back together, keep your fingers crossed and it should be fine!

This has fixed several laptops permanently and with this Advent I could go ahead and wipe the old Windows XP Home from it and install Ubuntu 8.04 that’s fantastic and  installs perfectly without any ‘geeking around’ necessary. My only problem is the fan control from linux, but that’s another story……

Posted by greatbear in Information Technology, 3 comments