It looks like we may see a successor to Adobe Photoshop’s content-aware fill very soon. NVIDIA has shown the latest in AI-driven technology, and that is one that reconstructs incomplete photos. The results are nothing short of outstanding.
Photoshop’s content-aware fill is the industry standard for removing unwanted objects in photos. The feature studies the pixels and patterns in images, and then fills up missing spots. However, it isn’t perfect, as color discrepancies and blurriness may crop up. Better results would often come hand-in-hand with expensive post-processing.
NVIDIA’s state-of-the-art deep learning technology takes a step towards mitigating these issues. The tool examines not only the image’s pixels to reconstruct the gaps, but determines how the image should look overall. The process, that the team led by NVIDIA’s Guilin Liu calls ‘image inpainting’, can “remove unwanted content, while filling it with a realistic computer-generated alternative.”
The image inpainting tool is trained under 55,116 masks of random shapes and holes to improve its performance. The AI learns by comparing cutouts to the original images to determine how to reconstruct missing pixels. Its accuracy is then verified under another 25,000 masks, without an original to compare.
The above image, from left to right, shows a disrupted photo, Adobe’s content-aware solution, NVIDIA’s image inpainting solution, and the original photo. From just a glance, one can already see that NVIDIA’s tool produces markedly better results than Adobe’s.
Paired with this announcement, NVIDIA shares a demonstration of their tool in action to the public. The video shows a person removing parts of an image. The AI then actively responds to these incomplete photos with a reconstruction. Impressively, it is even able to replace missing eyes without entering the uncanny valley.
In terms of shortcomings, NVIDIA does note that “one limitation of our method is that it fails for some sparsely structured images such as the bars on the door in Figure 11, and, like most methods, struggles on the largest of holes.”
Despite the revolutionary results shown, NVIDIA is not ready to release the tool just yet. The company has not disclosed information on when consumers, or even corporations, can get a hands-on with the tool.
More information on NVIDIA’s inpainting tool can be read from their research paper.
(Source: NVIDIA)
