Home > Photographic Imaging Software, Technology > Visual Content Recognition

Visual Content Recognition

December 8th, 2007

Visual Content Recognition is the ability to recognize and utilize content within the image. This is affecting every aspect of digital photography. The technology is already being used in numerous ways and the future is mind boggling. In the example above, face recognition detects triangles created by eyes and mouth, then sets focus and exposure. Let me give a couple examples and show some MIND BLOWING LINKS to examples and videos. You may want to look at the MIND BLOWING LINKS first.

STITCHING TOGETHER IMAGES: Photoshop can recognize pixels that match up and put them together. This allows you to stitch together multiple images to make one “bigger” image. Bigger means wider angle of view, more resolution, more dynamic range, or more depth of field. This also allows you to line up multiple hand held images and combine different elements together. Three shots of a group portrait can be aligned, then the eyes can be changed on the blinkers. Photoshop is making good use of Visual Content Recognition with Photomerg, Auto Align Layers, and the Auto Align Source Images when making HDR images. Microsoft has a technology called PhotoSynth which stitches together multiple images from multiple photographers. It will find all the images it can of a subject stitch them all together to make a 3D representation of it. For example: Photosynth could go to flicker and find all the tourist images of the Washington Monument from all angles, then stitch them into a 360 degree view. Mind blowing link number one is a Demo of Seadragon and Photosynth done at TED 2007 by Blaise Aguera Arcas. You can try out the Microsoft Live Labs Photosynth tech preview at http://labs.live.com/photosynth/

MIND BLOWING LINK #1 Seadragon and Photosynth Video

SEARCHING IMAGE ARCHIVES: Google searches currently aren’t even based on embedded metadata like keywords, that’s the next step. Skip forward to the near future when searches recognize the visual content. Looking for a picture of a “red background, two people, and a black dog”? Soon the big stock agencies like Corbis and Getty will have tools to search INSIDE the image for that content. Read more here: http://en.wikipedia.org/wiki/CBIR. Mind blowing link number two is an Idee Inc. application for searching for images based on predominant colors:

MIND BLOWING LINK #2 Multicolor Search Tool

SMART CROPING: This technology allows for a change in aspect ratio without cropping important information. It can take a horizontal image and make it a square without cropping or distorting. Smart Cropping looks for less important seams of pixels and removes them. For example: a vertical image with blue sky could be made into a square by removing seams of the sky. I think you really have to see this one to understand it. Mind Blowing link number three is a Video on Seam Carving done by Shai Avidan of Mitsubishi Electronic Research and Ariel Shamir of Interdisciplinary Center and Merl.

MIND BLOWING LINK #3 Smart Cropping Video

FACE RECOGNITION: Big time money is being spent on face recognition for identification of individuals. Casinos first used this to recognize problem gamblers. Now big time money is being spent on it to find terrorist and criminals.

Digital Cameras now have face recognition built in to the hardware. A camera can recognize the triangle created by two eyes and a mouth. Face recognition cameras can even recognize groups of people and use the information for focus and exposure. The distance to faces in a portrait is vital focus and flash information. Exposure metering for skin tones is also important. Automatic red eye correction is another Visual Content Recognition tool. The software finds the eyes and fixes them if they are red, but it doesn’t do much for the headache that sometimes accompanies those red eyes. Here’s some more info on how face recognition works: http://en.wikipedia.org/wiki/Facial_recognition_system

The technologies and links included above only scratch the surface of Visual Content Recognition will do photography technology. I look forward to reading comments on this article and the mind blowing links.

  1. gilberto
    December 28th, 2007 at 22:58 | #1

    wow, the smart cropping demo is pretty cool. It hurts my head to thinking about programming the seam detection and contruction

Comments are closed.