Google researchers recently discussed breakthroughs they’ve made in increasing image resolution. The scientists used a machine learning model to turn a low-res photo into a detailed high-res picture. It’s part of a growing trend of using AI to improve images. “We’re seeing a rise in AI-powered upscaling, especially in games, where technologies like NVIDIA DLSS use machine learning to recreate a much higher resolution image, which rivals, and sometimes exceeds the quality of the native images,” imaging expert Ionut-Alexandru Popa told Lifewire in an email interview. “This type of upscaling works great in computer games, where it manages to use fewer resources than rendering directly a higher quality image.”
Creating Pixel
Google has been exploring a way to upscale photos using a method called diffusion models. The company claims that this technique improves existing technologies when humans are asked to judge the results. One approach Google used is called SR3, or Super-Resolution via Repeated Refinement. “SR3 is a super-resolution diffusion model that takes as input a low-resolution image and builds a corresponding high-resolution image from pure noise,” Google researchers wrote in the blog post. “The model is trained on an image corruption process in which noise is progressively added to a high-resolution image until only pure noise remains.” Upscaling techniques aren’t new and are commonly used in photo editing applications, Popa said. “There are plenty of situations when you need a higher resolution image, so upscaling is used to create pixels in between existing ones,” he added. “Most people don’t realize, but when they watch TV on their 4K screen, the 1080p video signal is automatically upscaled to cover the entire screen. This is done automatically by your TV set.” Many current techniques are used to ‘guess’ the content of the new pixels so that the resulting image looks good, Popa said. “Currently, the most used algorithms for image upscaling are bilinear and bicubic methods, which ensure a continuous transition between adjacent pixels, with gradually changing color, but this method often results in loss of sharpness,” he added. “This is compensated partially by applying a sharpening pass over the upscaled image.” Image upscaling is crucial to how entertainment, media, and the internet works, photographer Sebastien Coell told Lifewire in an email interview. “For instance, rather than having to have multiple image sizes for a webpage, as in, one for use on a phone, one for a tablet,” he said, “if you can upscale that 1080p image to 2k or 4k and that phone image to a tablet and 1080p, you have suddenly reduced the number of images needed from 6 down to 2.” “You will also save the file space needed from the large 2k and 4k files so will reduce your overall file storage size needed by around 70-90%.”
Not Just for TV
Photo upscaling can also increase the quality of photos and could even help with medical imaging. Software developers say that upscaling can increase the resolution of images without any degradation in quality. But, Matic Broz, the founder of the photography website Photutorial, told Lifewire in an email interview that the actual results depend on the software used. “Recently, AI has found its way into image upscaling, although I’m not impressed with it so far,” he added. Broz said the best upscaling software he’s used is AI Image Enlarger by Vance AI. “Even their 8x image upscaler does not introduce any significant noise (that’s a 64x increase in resolution),” he said. “I expect the algorithms to only get better in the following years, allowing for even larger upscaling.” For still photographers, Broz said it’s an open question whether image upscalers are necessary. “Camera developers are constantly improving the resolution of camera sensors, now even with 100MP+ resolution,” he added. “Personally, I’ve used 24MP and about 50MP, and I’ve never felt the need for bigger images, not even for large prints.”
