Zoom and enhance!

With modern off-the-shelf AI tools, it's now possible to bring old or degraded photos back to life by adding (or correcting) colour, and enhancing faces.

02 Oct 2022 · 4 min · Mark Evans

Background

"Zoom and enhance!" is a movie trope that has long infuriated anyone who has even the most basic understanding of what is (and isn't) possible when manipulating images. In the past I often found myself shouting at the TV, “You can't recover information that was never there!”. It turns out that for movies and TV shows set before 2022 my outbursts were justified, but for those set during or after 2022, it's starting to look like I was very wrong.

Zoom and enhance!
— Rick Deckard (and every character from CSI)

With recent advances in AI—especially with regard to image interpretation, manipulation, and generation—a few new tools have become available online that remove a lot of the skill that was previously required to enhance and colourise old black & white, or colour degraded photos.

Below I provide links to 2 such AI tools and include 5 examples of old family photos that I've restored using them. I've also taken a recent colour photo, degraded it, and then used the same tools to see how the “restored” version compares with the unaltered original.

The restoration tools

For the face enhancement I’ve been using the Baseten application that uses the GFP-GAN algorithm (unfortunately Baseten have since removed this service). Try this alternative from Hugging Face instead.
For the colourisation I’ve been using the Palette.fm online app.

Examples

Below are 5 examples using my own family photographs that date between 1920 and 1979. Figures 1, 3, and 5 were originally black & white. While Figure 2 was originally colour, but the colour has degraded considerably and now has a very strong red cast. Figure 4 is a duotone. Each photo was digitised by scanning the original print (not the negative).

Two young boys on a ladybird ride. Showing a blurry original version, and a restored full colour version. — My brother (right) and me (left) riding a ladybird carousel, circa 1979. Click or tap the image to toggle between original and restored images.

A red cast image of a young boy and his grandfather. Showing the original version, and a restored full colour version. — My brother and paternal grandfather (aka bampi), circa 1975. 2 Torrington Rd, Gendros, Swansea, UK. Click or tap the image to toggle between original and restored images.

A young girl and her monther in black and white. And a full colour restored version — My mom and her mother, circa 1949. Click or tap the image to toggle between original and restored images.

A portrait photograph of a girl in her early twenties. The image is a duotone photograph and is out of focus. It's shows with a sharp full colour restored version. — Edith Lilian Evans (nee Mabbett, born 1899), my great-grandmother, circa 1920. Click or tap the image to toggle between original and restored images.

A black and white photograph of a baby in a bathtub located in a garden. Also shown is the full colour restored version. — Gillian Heinson (my mom's cousin), circa 1947. Click or tap the image to toggle between original and restored images.

Are the results “real”?

The results in the examples above look amazing, but are they real? i.e. are they an accurate representation of the original photos’ subjects? We can't know for sure, as when restoring old photos the only source of truth we have are the old photos themselves. But what if we used a modern photo, removed the colour, and added a blur to the image in an attempt to make it look old? How closely would the results of face enhancement and colour restoration match the original image?

For this experiment I used an image of the wife and me that was taken a couple of weeks ago. I then desaturated the image and applied a 2 px blur to remove enough of the details such that it started to look like the old photos in the examples above (actually, I think it looks worse).

The original image is shown below [fig. 6]. The degraded image is shown in fig. 7, you can click or tap on fig. 7 to see the result of the restoration.

A recent photo of my wife and I in colour, standing in front of a bush — The good lady her indoors, and me.

A degraded version of the recent photo of my wife and I in colour, standing in front of a bush, also shown is the restored version — A degraded modern image, and the results of the restoration process. Click or tap the image to toggle between original and restored images.

There are some pink flowers in the bush that should be green, and the face enhancement has decided to shave my beard off, but apart from that it's pretty good. Is it as good as the original? of course not. The key point here is that it's substantially better than the degraded image, and it's close enough to the original that it doesn't look like the faces or colour are "made up".

Conclusion

Some people might look at this type of AI photo restoration as some form of “distortion of the truth”, but I'd argue that an old faded black & white photograph is not the truth, it's a degraded version of the best “truth” that could be captured at the time. Does an AI colourised photo represent a perfect reproduction of the true colour? no, but guess what—the world was never black & white either.

I've always felt very disconnected from the people and events shown in old black & white photographs—even when they were my own family members—I didn’t understand how big the disconnect was until I saw the work of Marina Amaral, especially her 'Faces of Auschwitz' project. Seeing those manually colourised photos made the events and people appear more real—which is extremely important given the events and people that they record.

I'm sure that these AI photo restoration tools will soon become part of the many online photo libraries (such as Google Photos, Amazon Photos, and Microsoft's OneDrive), and I hope they do, as that will mean they are available to the general public and hopefully no specialised knowledge will be required.