Dr. Tim Harris, senior software engineer at Digital Vision, looks at how to go about the restoration of archive material
Film archives and content owners are today faced with the issue of how best to preserve and potentially monetise their film and tape-based assets. With restoration and re-mastering technologies, broadcasters and film archivists can breathe new life into archives and repurpose material for digital cinema, HDTV and DVD/Blu-ray platforms.
Restoration of archive material is, however, a time consuming manual process that requires both retouching and quality assurance conducted by an expert operator. Manufacturers are looking at ways to simplify this process and it will soon be possible for content owners to use intelligent software algorithms to analyse material and make repairs automatically.
The restoration market is stuck as a cottage industry due to the fundamental economics of the processes involved in restoration. TV programmes and movies created and stored in an archive can be given a new lease of life through restoration, but since restoration isn’t free, ROI calculations need to be undertaken when evaluating whether material can be restored.
With very popular film material, significant time and effort can go into painstakingly restoring the content. With less popular material, or material which is longer in duration, less time and effort may be available to restore the content. With very obscure material, the ROI calculations do not tally and the material cannot be restored economically.
One way to evaluate the efficiency of a restoration workflow is to look at its throughput. For a film like Blade Runner, over ten seconds was spent restoring each frame of the movie, whereas for content for an old TV series the restoration throughput would be in the region of ten frames per second. The workflow required for both restorations is similar, so the difference resides in the amount of time spent on each stage. It is also worth noting that the workflow for audio and video are very similar as well, though the workload split between the two is dependent on the project being undertaken.
If we can reduce the cost of restoring a piece of material, then more material becomes available for restoration. By combining the audio and video pipeline, adding automation, and improving the throughput of each stage, we can make restoration accessible to a much broader range of content.
Hardware or software?
When designing a restoration system, an important choice is whether to use external hardware or a software-only system. Hardware tools provide a guaranteed throughput and industry-leading algorithms, but are updated rarely and require highly skilled operators to use. These hardware tools are well-suited to environments with large amounts of SDI infrastructure and workflows that involve VTRs. Video and audio restoration both benefit from a software infrastructure. Audio restoration in hardware, like video restoration in hardware, involves real-time audio playout through AES/EBU using hardware boxes with fixed parameter settings for declicking and decrackling.
In a file-based workflow, there are a number of disadvantages to using custom hardware. The extra playout and ingest hardware required comes with a cost overhead and a significant reliability hit. If there is a problem partway through the processing, a hardware system will require a complete replay as the result is non-repeatable. As such, to achieve a consistent look throughout a clip, the entire clip would need to be reprocessed after a single frame-drop.
Hardware solutions tend not to be portable, and require specialist maintenance, unlike commodity computing hardware, which is much more widely deployed. The biggest drawback though with hardware-only systems is the lack of metadata and logging output on the processing of a job.
The work done to a piece of material and the settings used during processing are untraceable without a significant effort by the human operator. Following the Hollywood trend, many European broadcasters are looking towards JPEG 2000 as their archive codec of choice for high value assets.
JPEG 2000 allows parallel processing of the audio, video and metadata components and delivery in a standard structure that can interface with MAMs, content servers and archive systems.
There has been wider adoption of JPEG 2000 in MXF including PrestoSpace, EDCINE, US Library of Congress, and SAMMA Systems. The existing commercial implementations from SAMMA have led the way in the adoption of JPEG 2000 in MXF.
Restoration and archiving is not well served by the existing MXF application-specific standards. Customising an existing standard within its own framework would be a pragmatic technique to ensure greater flexibility and lower costs to archive and restoration customers.
Improved algorithms for detecting and removing tape and compression artefacts are required. Distributed, or cloud-based, processing of material can enable dynamic scaling of a restoration infrastructure to meet the client’s needs quickly.
MEETING THE CHALLENGES
The issues outlined in this feature are being addressed by Digital Vision as part of an EU-funded project (VideoStar) in conjunction with Studio Hamburg and Cube-Tec. The VideoStar project will see Digital Vision develop an automatic audio and video solution that will enable content owners to create a clean digital file for every asset in their archive.
As well as the cleaned file, which will contain all the metadata related to changes made, the digital archive will also contain the ingested RAW media and metadata on the source, processes applied and information from the QC analysis.
VideoStar will use MXF, JPEG 2000, Broadcast Wave and XML file formats, and new workflows combined with improved restoration algorithms and technologies.
Digital Vision and Cube-Tec are working together to develop two new products that aim to fill the gap between completely manual restoration and fully automated, operator-less working. Digital Vision will be demonstrating elements of the project later in the year.
Digital Vision was founded in 1988 in response to the need for a high-quality grain and noise reducer for telecine applications. Its Phoenix range of products is widely used for film and video restoration.