New Google Brain research brings the ‘zoom and enhance’ trope to reality

Re-creating a pixelated image is the holy grail of impossible CSI technology. And while zooming in on grainy footage, enhancing the image, and seeing the culprit reflected in the victim’s iris is still the stuff of fantasy, some new research from Google Brain may have done the next best thing, using a pair of neural networks to process a 8 pixel x 8 pixel image and generate an approximation of the original.

To be clear right off the bat: this isn’t some magical sharpening of the image — the faces it generates aren’t those of the initial person, but rather a fictional creation that represents the computer’s best guess at approximating the original. As Ars Techinca breaks down, the software utilizes a two different neural networks to generate the images. The first is a “conditioning network,” which maps the pixels of the low-resolution picture to a similar high-resolution one that gets used as a rough skeleton of how the face or room should look. The second is a “prior network” analyzes the pixelated image image and tries to add details based on existing images with similar pixel locations. The two outputs are then combined to create the (fairly impressive) final image. Full details on the process can be found in the published research paper here, for those looking for a deeper dive into the methodology.

It’s not the first time we’ve seen AI used to manipulate images this way, either. Neural networks are getting increasingly advanced when it comes to generating images, and both Google and Twitter already have put effort into sharpening video and still images online through their respective RAISR and Magic Pony projects. And who knows? At the rate technology is advancing, maybe one day being able to truly zoom and enhance might just be possible after all.

by Chaim Gartenberg