Scanning Old Photos

Although I now use a digital camera, I still have piles and piles of old chemical photos sitting around decomposing. I'd like to scan these old photos for easy distribution and archival purposes.

The problem is the sheer number of photos--several hundred, with more on the way. Much of the work of dealing with scanned photos is just rotating and cropping, and I felt these tedious tasks could actually be automated. I know of no existing program, commercial or otherwise, to do this; so I wrote one.

The input to the program is a single, large scan of several photographs. The photographs are intended to have any size and orientation, although the standard aspect ratio is important. The output of the program is the individual photographs, properly rotated and cropped from the background.

This sounds like an insurmountable problem in machine vision, since even the best research codes cannot recognize the boundaries of completely unconstrained scenes. So, as usual, we just cheat--rather than try to recognize the images, which are arbitrary; we just try to recognize the background, which is known and fixed. That is, we look the for scanner's backplate, which should surround each photo.

My scanner's backplate is a nearly black, textured piece of plastic. It's not the ideal surface to recognize, since photos can occasionally contain quite similar-looking regions, but it's possible. I recognize backplate in a small patch based on the patch's mean gray level (dark), pixel variance (fairly large), and color content (very little). The output of the filters for each of these three properties are shown in the red, green, and blue channels of the "detectors" image, below. Even though none of these filters are very good individually; they tend to fail in different locations, so by requiring all the detectors to agree (by taking the product of their outputs), I get the nice result shown in "classified".

Source: The image from the scanner (reduced here from the 2400x3096 pixel original)

Detectors: The output of the pixel value, variance, and color detectors shown as red, green, and blue.

Classified: The product of the three detectors.

Now that I can reliably separate foreground (pictures) from background, I need to find the boundaries of and extract each picture. I used a simple region-growing approach, where I drag around the corners of a quadrilateral. By pulling out corners that lie on foreground parts of the image, and pushing in corners that lie on background parts of the image, I can occasionally grab an entire photo. By throwing out regions that have an unreasonable size, bad aspect ratio (far from 3:2), or non-square corners, I eventually end up covering all the pictures in the scan. See "Regions", below.

Now that we know the corners of each photo, the individual pictures can easily be extracted from the scan. For now, I don't try to automatically figure out which direction should be "up" to properly rotate the resulting photo--I can't think of a heuristic that works more often than random chance. Just to be clear, the "Final" result below was obtained completely automatically from the original source scan--there was zero human intervention, guidance, or retouching (just the way I like it!).

Regions: The quadrilaterals that eventually grew to cover each photo.

Final Images: Note how each image is nicely aligned, even when the original scan was slightly crooked.

For more examples, see parts of my photo archive: Glennallen 1980's, Glennallen to 1995, and Glennallen to 2001.

In total, my little program properly extracted 140 images from 64 scans; 2 images were extracted incorrectly and required manual fixing--writing the program actually turned out to be a timesaver! The only reason it's not a commercial product right now is that scanner backplates vary so much. I suspect most scanners with white backplates would require a completely different approach to separate figure from ground. It could probably be done, but it'd have to do a pretty darn good job to be useful.

Finally, it's commonly held that digital photos are a good deal less reliable as a long-term storage solution, because digital formats and media are complex and change frequently. Since I'm a programmer, I have a heaping helping of digital hubris; so I tend not to believe the naysayers. I also have good, scriptable C code to read JPEG's, so I think the photos will be safe for a few dozen years at least.