PAL Colour Recovery

by W.A. Steer PhD
Back to contents About...

PAL Colour Recovery from black-and-white 'telerecordings'

Numerous BBC television programmes of cult and classic status from the late 1960's and 1970's which were originally made in colour now only exist in the form of black and white 'telerecordings'. These programmes include a number of episodes of "Doctor Who", "Steptoe and Son", and "Dad's Army". It has long been believed that we'd never again see these programmes in their original colour. Intriguingly however, for technical reasons, most of these black and white recordings actually contain remnants of the PAL colour-coding signal - but until recently it was not thought possible to decode them. Through the Colour Recovery Working Group I have been working on this problem and in the past month have demonstrated practical image-processing colour-recovery approaches.

Introduction

Numerous BBC television programmes of cult and classic status from the late 1960's and 1970's which were originally made in colour on videotape now only exist in the form of black and white 'telerecordings'. These programmes include a number of episodes of "Doctor Who", "Steptoe and Son", and "Dad's Army". In the old days, videotape was extremely expensive, and therefore there was a strong temptation to reuse it. Actors' unions imposed limits on repeat-showings of programmes, to ensure that actors were not done out of their jobs. One way and other the BBC had various policies of junking videotapes, which continued until 1978. Many programmes were lost forever while others exist only as black-and-white 'tele-recordings' (essentially a high-quality black-and-white 16mm film recording of a black-and-white monitor) which were made by the BBC for the export market.

It has long been believed that we'd never again see these programmes in their original colour. Intriguingly however, for technical reasons, most of these black and white recordings actually contain remnants of the PAL colour-coding signal - so-called 'chromadots', but until recently it was not thought possible to decode them.

As I explain in slightly more detail on my earlier PAL decoding pages (Software-based PAL colour decoding), colour for traditional analog television systems is encoded as modulated 4.43MHz subcarrier which lies on top of an essentially black-and-white picture. This encoded colour appears as very fine cross-hatching if the colour TV signal is watched on a simple black-and-white monitor. This same cross-hatching or 'chromadots' are also present on many tele-recordings of these old programmes.

Quoting from my Software-based PAL colour decoding page:

The transmitted colour TV signal is effectively a black-and-white picture but with areas of fine patterning the strength of which is proportional to the saturation of the colour portrayed. Of note, U-axis colours (pure yellows or blues) appear as diagonal stripes \\\, while V-axis colours (pure reds or greens) appear as diagonals ///. Mixture colours, such as orange (red+yellow) give rise to a superposition of the two, ie cross-hatching. These patterns can be seen on a black-and-white monitor fed with a colour TV signal. (See figure below)

Raw PAL-coded TV picture (1 field). Note the colour phase-reference 'burst', left, and the fine diagonal stripes and cross-hatching which code the colour - compare with colourised insert.

It is not just the pattern of these 'chromadots' but also their phase (their absolute position in the picture) which is needed to decode the colour. In a normal TV signal this is quite easy, but with the telerecordings the usual phase-reference 'burst' (which exists off the left-margin of normal TV pictures) is not recorded. Furthermore because of the recording off a cathode-ray-tube TV screen the geometry of the picture is distorted, again making it almost impossible to establish the exact phase of the signal at any arbitrary place on the image. For these reasons, most engineers would not even contemplate the possibility of recovering the colour from the telerecordings.

James Insell, archivist at the BBC (formerly of BBC R&D), on the other hand has clearly found these remnants of colour-coding tantalising, and for some years has been trying to generate interest in decoding the colour from these old recordings. In April 2007, he formed the Colour Recovery Working Group as a Wikispace, and a year later (via a round-about route) I stumbled across it.

Having worked (as a hobby) on PAL decoding almost 10 years ago, I felt that if anyone could recover the colour from these films, I could. I joined the Colour-Recovery group, obtained some source images, and within a week or two had some very nice demonstrations of (partially-) recovered colour.

For the full background, please see: http://colour-recovery.wikispaces.com/

There was also an article in the Guardian newspaper, at http://www.guardian.co.uk/technology/2008/mar/06/research.bbc which helped to bring this Colour Recovery idea to a broader audience. The article mistakenly says that this process is already being used - this was absolutely not true at the time the article was written. The process was extremely experimental at that stage, but did show some promise that "something" was there.

My main contributions (in blog-form) are:

Summary

Jonathan Wood and James Insell initially provided me with just two still frames from an experimental high-definition (1920x1080 pixel) telecine of the original negative from an old telerecording. These were from an episode of Top of the Pops from 1970. I later found out that the whole 45-minute programme exists on the black-and-white film 'telerecording', but that only about 15-minutes-worth of the programme is still available from the original colour videotape.

This (now historical) article was written in year 2008, about colour-recovery development work performed in the same year. With the benefit of hindsight, it is unfortunate and regrettable that the images used for the initial work described in this article (below) show a TV presenter whose despicable true character has since come to light (2011-12). However, the particular section of telerecording and video from ToTP 1970, had technical characteristics without which the whole colour-recovery ball may never have got rolling. We cannot re-write history.

Image from black-and-white telerecording
Similar frame from remaining colour videotape

1:1 pixel crop of HD scan of black-and-white telerecording, showing embedded chroma signal / 'chromadots'

Besides the geometric distortions and lack of subcarrier phase-reference already mentioned, even in the high-definition telecine the adjacent scanlines of the original recording merge into one another. These lines partially merge because of a 'spot wobble' process originally employed to reduce the visibility of the line structure when the telerecording was made. Further, the original full frame (two interleaved interlaced fields) comprises 576 scanlines, but when scanned at 'only' 1080 lines aliassing occurs and (even theoretically, because 1080 is less than twice 576) the lines cannot be fully resolved. This adds further complications to the colour recovery. If the recovery process is developed to the point of viability for programmes then it is hoped that higher resolution scans (1920x1440 or 1920x1556) could be made.

Results

10 April 2008

Typical result of doing a totally naive PAL-decode on the film image. The phases and therefore the colours are all wrong.

11 April 2008

A slighly better decode, which uses vertical chroma filtering to separate U and V components, yielding an improvement in the recovered colours. The hue still shifts across the image, and this kind of result is of little practical use.

12 April 2008

I used a trick to force all colours to decode to the -U/+V colour-quadrant. This recovers flesh-tones and red-orange-yellow-olivegreen colours, and gives consistent colours across the image. It works remarkably well for this particular image, but other colours are consistently incorrectly recovered; blue and purple are rendered as a dull pea-green for example.

The colour saturation is still somewhat off in this image, traced to various minor errors in my programming.

Following various refinements, and bug-fixes, the same basic procedure was giving quite useful results by late April, and a 34-second movie sequence was processed.

5 May 2008

PAL Colour-Recovery from 16mm black-and-white telerecording. This is still restricted in colour gamut, but the overall quality has been improved.
I have developed some simple (but believed-novel) techniques of reducing PAL cross-colour, also employed in this example.

Oddly, the hue of the big square around the "1" (which should be orange) is still too red, and possibly the flesh tones also are erring on the pink/red side. At the time of writing (9 May), I'm still not sure what the source of these errors are (although I have some ideas). I don't see this as a serious problem at the moment.

This is frame 750 from the test sequence, processed as the above image. Although impressive, the restricted colour-gamut is more apparent here. The shirt of the guy at the left of the picture should be dark blue, but instead is slightly greeny. Similarly the jeans of the two guys on the right should be dark blue denim, and their waistcoats a sky-blue. The mirrors in the background on the top right should have strongly purple reflections (but again come out as mushy green).

Next steps

The next steps really are to recover absolute phase of the chroma signal (which effectively means mapping the distortion of the image geometry one way or another) to enable the original full gamut of colours to be recovered.

Richard Russell (see the Wikispace) has made good progress on this front, but I hope to make my own attempts soon. There's plenty of scope for improving the robustness of the full-gamut algorithms (in contrast, the restricted-gamut process shown above is very reliable, with no nasty surprises).

Acknowledgements

James Insell (BBC Archives / R&D) - without whom the project would not have happened and I would never have been aware of it.
Jonathan Wood (telecine and colourist at the BBC) - who performed the special high-definition film scans
...

Created: May 2008
Last modified: 9 May 2008
Source: http://www.techmind.org/colrec/index.html