Comparison gallery of image scaling algorithms

From HandWiki
Short description: Visual examples of techniques for image scaling

This gallery shows the results of numerous image scaling algorithms.

Upscaling methods

An image size can be changed in several ways. Consider resizing a 160x160 pixel photo to the following 40x40 pixel thumbnail and then upscaling the thumbnail to a 160x160 pixel image. Also consider doubling the size of the following image containing text.

Low-resolution images
Thumbnail Text
Thumbnail Image Original Image 40x40 pixel thumbnail
Comparison of upscaling methods
Original photo Upscaled thumbnail Upscaled text Algorithm and description
160×160 thumbnail reference Nearest-neighbor interpolation Nearest-neighbor interpolation

One of the simpler ways of increasing the size, replacing every pixel with a number of pixels of the same color. The resulting image is larger than the original, and preserves all the original detail, but has (possibly undesirable) jaggedness. The diagonal lines of the "W", for example, now show the "stairway" shape characteristic of nearest-neighbor interpolation. Other scaling methods below are better at preserving smooth contours in the image.[citation needed]

160×160 thumbnail reference Bilinear interpolation Linear Interpolation

Linear (or bilinear, in two dimensions) interpolation is typically good for changing the size of an image, but causes some undesirable softening of details and can still be somewhat jagged.[citation needed]

160×160 thumbnail reference Bicubic Interpolation Cubic Interpolation

Better scaling methods include Lanczos resampling and Mitchell-Netravali filters.[citation needed]

160×160 thumbnail reference Fourier-based interpolation Fourier-based Interpolation + saturation

Fourier-based interpolation

Simple Fourier based interpolation based on padding of the frequency domain with zero components (a smooth-window-based approach would reduce the ringing). Beside the good conservation of details, notable is the ringing and the circular bleeding of content from the left border to right border (and way around).[citation needed]

160×160 thumbnail reference File:40 by 40 thumbnail of 'Green Sea Shell' (x4 DCCI).png File:Wiki dcci 2x.png

Edge-directed interpolation algorithms aim to preserve edges in the image after scaling, unlike other algorithms which can produce staircase artifacts around diagonal lines or curves. Examples of algorithms for this task include:

160×160 thumbnail reference hq4x scaling hq2x scaling

Pixel art scaling algorithms (hqx)

For magnifying computer graphics with low resolution and few colors (usually from 2 to 256 colors), better results will be achieved by pixel art scaling algorithms such as hqx or xbr. These produce sharp edges and maintain high level of detail. Unfortunately due to the standardized size of 218x80 pixels, the "Wiki" image cannot use HQ4x or 4xBRZ to better demonstrate the artifacts they may produce such as row shifting.


The example images use HQ4x and HQ2x respectively.

160×160 thumbnail reference 160 by 160 upscaled thumbnail of 'Green Sea Shell' (4xBRZ) Image after scaling (2xBRZ)

Pixel art scaling algorithms (xbr)

The xbr family is very useful for creating smooth edges. It will however deform the shape significantly, which in many cases creates a very appealing result. However it will create an effect similar to posterization by grouping together local areas into a single colour. It will also remove small details if in-between larger ones which connect together.

The example images use 4xBRZ and 2xBRZ respectively.

160×160 thumbnail reference160×160 thumbnail reference 160 by 160 upscaled thumbnail of 'Green Sea Shell' (GemCutter Preserve Details)160 by 160 upscaled thumbnail of 'Green Sea Shell' (Smooth Edges) Image after scaling (GemCutter Preserve Details)


Image-after-scaling smooth

Pixel art scaling algorithms (GemCutter)

An adaptable technique which can deliver variable amounts of detail or smoothness. It aims to preserve the shape and coordinates of original details, without blurring those details into neighboring ones. It will avoid blending pixels which directly touch each other, and instead only blend pixels with their diagonal neighbors.

The "Cutter" name comes from its tendency to cut corners of squares and turn them into diamonds, as well as create distinct faces along stair-stepped pixels, i.e. those which exist on along the angles of edges found on a diamond. The "Gem" prefix both refers to the diamond cut, and also many traditional gem cuts which involve cutting corners at a 45-degree angle.


The example images use GemCutter Preserve Details (Top), and GemCutter Smooth Edges (Bottom).

160×160 thumbnail reference Vectorization to 48 colors (Inkscape) Vectorization

Vectorization first creates a resolution-independent vector representation of the graphic to be scaled. Then the resolution-independent version is rendered as a raster image at the desired resolution. This technique is used by Adobe Illustrator Live Trace, Inkscape, and several recent papers.[5] Scalable Vector Graphics are well suited to simple geometric images, while photographs do not fare well with vectorization due to their complexity.


Note that the special characteristics of vectors allow for greater resolution example images. The other algorithms are standardized to a resolution of 160x160 and 218x80 pixels respectively.

160×160 thumbnail reference EDSR waifu2x (unknown version?)

Deep convolutional neural networks

Using machine learning, convincing details are generated as best guesses by learning common patterns from a training data set. The upscaled result is sometimes described as a hallucination because the information introduced may not correspond to the content of the source. Enhanced deep residual network (EDSR) methods have been developed by optimizing conventional residual neural network architecture.[6] Programs that use this method include waifu2x, Imglarger and Neural Enhance.

160×160 thumbnail reference ESRGAN alt=RealESRGAN-x4plus TTA

Deep convolutional neural networks using perceptual loss

Developed on the basis of the super-resolution generative adversarial network (SRGAN) method,[7][better source needed] enhanced SRGAN (ESRGAN)[8] is an incremental tweaking of the same generative adversarial network basis. Both methods rely on a perceptual loss function[9] to evaluate training iterations.

Examples of enlarged images

Below are examples of various images enlarged 4x using each scaling algorithm.

Examples of various images enlarged 4x
Scaling algorithm Portrait photo[* 1] Text[* 2] Pixel art[* 3] Anime art[* 4]
Nearest-neighbor interpolation 170px 170px 170px 170px
Bilinear interpolation 170px 170px 170px 170px
Bicubic interpolation
(a=−0.5)
170px 170px 170px 170px
Lanczos resampling 170px 170px 170px 170px
Deep convolutional neural networks using perceptual loss(Real-ESRGAN[* 5]) 170px 170px 170px 170px

Downscaling methods

Consider resizing a 600×600 pixel image to 150×150.

Comparison of downscaling methods
Original photo[* 6] Downscaled Algorithm and description
150x150px 150x150px
150x150px 150x150px
150x150px 150x150px
150x150px 150x150px

Notes

  1. ↑ https://dome.mit.edu/handle/1721.3/195767 enlarged 4x, cropped at x:420, y:210, w:170, h:200
  2. ↑ :File:Image-before-scaling.png outer perimeter 2px, tone -25%, enlarged 4x, cropped at x:75, y:0, w:170, h:192
  3. ↑ :File:Pixelart-tv-iso.png enlargemed 4x, cropped at x:25, y:15, w:170, h:200
  4. ↑ :File:Wikipe-tan face.svg 128px PNG file, enlarged 4x, cropped at x:110,y:80,w:170,h:200
  5. ↑ https://github.com/xinntao/Real-ESRGAN model: realesrgan-x4plus for portrait photo, realesrgan-x4plus-anime for others
  6. ↑ Resized by the web browser own algorithm, open it for the full size original image.

References

  1. ↑ Orchard, M.T.; Li, Xin (September 2000). "New Edge Directed Interpolation". Internation Conference on Image Processing. 2. Vancouver, BC, Canada. pp. 311-4. doi:10.1109/ICIP.2000.899369. ISBN 0-7803-6297-7. http://www.csee.wvu.edu/~xinl/papers/ICIP2000a.pdf. Retrieved 2016-07-03. 
  2. ↑ Zhang, D.; Xiaolin Wu (2006). "An Edge-Guided Image Interpolation Algorithm via Directional Filtering and Data Fusion". IEEE Transactions on Image Processing 15 (8): 2226–38. doi:10.1109/TIP.2006.877407. PMID 16900678. Bibcode: 2006ITIP...15.2226Z. 
  3. ↑ Zhou, D.; Shen, X.; Dong, W. (24 August 2012). "Image Zooming Using Directional Cubic Convolution Interpolation (Paper and Code)". IET Image Processing 6 (6): 627–634. doi:10.1049/iet-ipr.2011.0534. https://www.mathworks.com/matlabcentral/fileexchange/38570-image-zooming-using-directional-cubic-convolution-interpolation. Retrieved 13 September 2015. 
  4. ↑ Shaode Yu; Rongmao Li; Rui Zhang; Mou An; Shibin Wu; Yaoqin Xie (2013). "Performance evaluation of edge-directed interpolation methods for noise-free images". arXiv:1303.6455 [cs.CV].
  5. ↑ Johannes Kopf and Dani Lischinski (2011). "Depixelizing Pixel Art". ACM Transactions on Graphics 30 (4): 99:1–99:8. doi:10.1145/2010324.1964994. 
  6. ↑ Lim, Bee; Son, Sanghyun; Kim, Heewon; Nah, Seungjun; Kyoung Mu Lee (2017). "Enhanced Deep Residual Networks for Single Image Super-Resolution". arXiv:1707.02921 [cs.CV].
  7. ↑ "Generative Adversarial Network and Super Resolution GAN(SRGAN)". 26 April 2020. https://medium.com/@peaceful0907/generative-adversarial-network-and-super-resolution-gan-srgan-e2e7a4440066. Retrieved 26 August 2020. 
  8. ↑ Wang, Xintao; Yu, Ke; Wu, Shixiang; Gu, Jinjin; Liu, Yihao; Dong, Chao; Chen Change Loy; Qiao, Yu; Tang, Xiaoou (2018). "ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks". arXiv:1809.00219 [cs.CV].
  9. ↑ "Perceptual Loss Functions". 17 May 2019. https://deepai.org/machine-learning-glossary-and-terms/perceptual-loss-function. Retrieved 26 August 2020. 

{{Navbox |name = Video processing |title = Video processing |state = autocollapse |listclass = hlist |basestyle = text-align:center;

|group1 = Post-processing |list1 =

|group2 = Special processing |list2 =

|group3 = 24 to 30 fps conversion |list3 =

|group4 = 30 to 24 fps conversion |list4 =

}}