Frame rate

From HandWiki
Short description: Number of frames rendered in one second


Frame rate (commonly expressed in frames per second or FPS) is typically the frequency (rate) at which consecutive images (frames) are captured or displayed. This definition applies to film and video cameras, computer animation, and motion capture systems. In these contexts, frame rate may be used interchangeably with frame frequency and refresh rate, which are expressed in hertz. Additionally, in the context of computer graphics performance, FPS is the rate at which a system, particularly a GPU, is able to generate frames, and refresh rate is the frequency at which a display shows completed frames.[1] In electronic camera specifications frame rate refers to the maximum possible rate frames could be captured, but in practice, other settings (such as exposure time) may reduce the actual frequency to a lower number than the frame rate.

Human vision

Further information: Philosophy:Motion perception

The temporal sensitivity and resolution of human vision varies depending on the type and characteristics of visual stimulus, and it differs between individuals. The human visual system can process 10 to 12 images per second and perceive them individually, while higher rates are perceived as motion.[2] Modulated light (such as a computer display) is perceived as stable by the majority of participants in studies when the rate is higher than 50 Hz. This perception of modulated light as steady is known as the flicker fusion threshold. However, when the modulated light is non-uniform and contains an image, the flicker fusion threshold can be much higher, in the hundreds of hertz.[3] With regard to image recognition, people have been found to recognize a specific image in an unbroken series of different images, each of which lasts as little as 13 milliseconds.[4] Persistence of vision sometimes accounts for very short single-millisecond visual stimulus having a perceived duration of between 100 ms and 400 ms. Multiple stimuli that are very short are sometimes perceived as a single stimulus, such as a 10 ms green flash of light immediately followed by a 10 ms red flash of light perceived as a single yellow flash of light.[5]

Film and video

Silent film

Early silent films had stated frame rates anywhere from 16 to 24 frames per second (fps),[6] but since the cameras were hand-cranked, the rate often changed during the scene to fit the mood. Projectionists could also change the frame rate in the theater by adjusting a rheostat controlling the voltage powering the film-carrying mechanism in the projector.[7] Film companies often intended that theaters show their silent films at higher frame rates than they were filmed at.[8] These frame rates were enough for the sense of motion, but it was perceived as jerky motion. To minimize the perceived flicker, projectors employed dual- and triple-blade shutters, so each frame was displayed two or three times, increasing the flicker rate to 48 or 72 hertz and reducing eye strain. Thomas Edison said that 46 frames per second was the minimum needed for the eye to perceive motion: "Anything less will strain the eye."[9][10] In the mid to late 1920s, the frame rate for silent film increased to 20–26 FPS.[9]

Sound film

When sound film was introduced in 1926, variations in film speed were no longer tolerated, as the human ear is more sensitive than the eye to changes in frequency. Many theaters had shown silent films at 22 to 26 FPS, which is why the industry chose 24 FPS for sound film as a compromise.[11] From 1927 to 1930, as various studios updated equipment, the rate of 24 FPS became standard for 35 mm sound film.[2] At 24 FPS, the film travels through the projector at a rate of 456 millimetres (18.0 in) per second. This allowed simple two-blade shutters to give a projected series of images at 48 per second, satisfying Edison's recommendation. Many modern 35 mm film projectors use three-blade shutters to give 72 images per second—each frame is flashed on screen three times.[9]

Animation

This animated cartoon of a galloping horse is displayed at 12 drawings per second, and the fast motion is on the edge of being objectionably jerky.

In drawn animation, moving characters are often shot "on twos", that is to say, one drawing is shown for every two frames of film (which usually runs at 24 frame per second), meaning there are only 12 drawings per second.[12] Even though the image update rate is low, the fluidity is satisfactory for most subjects. However, when a character is required to perform a quick movement, it is usually necessary to revert to animating "on ones", as "twos" are too slow to convey the motion adequately. A blend of the two techniques keeps the eye fooled without unnecessary production cost.[13]

Animation for most "Saturday morning cartoons" was produced as cheaply as possible and was most often shot on "threes" or even "fours", i.e. three or four frames per drawing. This translates to only 8 or 6 drawings per second respectively. Anime is also usually drawn on threes or twos.[14][15]

Modern video standards

Due to the mains frequency of electric grids, analog television broadcast was developed with frame rates of 50 Hz (most of the world) or 60 Hz (Canada, US, Japan, South Korea). The frequency of the electricity grid was extremely stable and therefore it was logical to use for synchronization.

The introduction of color television technology made it necessary to lower that 60 FPS frequency by 0.1% to avoid "dot crawl", a display artifact appearing on legacy black-and-white displays, showing up on highly-color-saturated surfaces. It was found that by lowering the frame rate by 0.1%, the undesirable effect was minimized.[original research?]


At its native 24 FPS rate, film could not be displayed on 60 Hz video without the necessary pulldown process, often leading to "judder": To convert 24 frames per second into 60 frames per second, every odd frame is repeated, playing twice, while every even frame is tripled. This creates uneven motion, appearing stroboscopic. Other conversions have similar uneven frame doubling. Newer video standards support 120, 240, or 300 frames per second, so frames can be evenly sampled for standard frame rates such as 24, 48 and 60 FPS film or 25, 30, 50 or 60 FPS video. Of course these higher frame rates may also be displayed at their native rates.[16][17]

Electronic camera specifications

In electronic camera specifications frame rate refers to the maximum possible rate frames that can be captured (e.g. if the exposure time were set to near-zero), but in practice, other settings (such as exposure time) may reduce the actual frequency to a lower number than the frame rate.[18]

Frame rate up-conversion

Frame rate up-conversion (FRC) is the process of increasing the temporal resolution of a video sequence by synthesizing one or more intermediate frames between two consecutive frames. A low frame rate causes aliasing, yields abrupt motion artifacts, and degrades the video quality. Consequently, the temporal resolution is an important factor affecting video quality. Algorithms for FRC are widely used in applications, including visual quality enhancement, video compression and slow-motion video generation.[citation needed]

Low frame rate video
Video with 4 times increased frame rate

Methods

Most FRC methods can be categorized into optical flow or kernel-based[19][20] and pixel hallucination-based methods.[21][22]

Flow-based FRC

Flow-based methods linearly combines predicted optical flows between two input frames to approximate flows from the target intermediate frame to the input frames. They also propose flow reversal (projection) for more accurate image warping. Moreover, there are algorithms that gives different weights of overlapped flow vectors depending on the object depth of the scene via a flow projection layer.

Pixel hallucination-based FRC

Pixel hallucination-based methods use deformable convolution to the center frame generator by replacing optical flows with offset vectors. There are algorithms that also interpolates middle frames with the help of deformable convolution in the feature domain. However, since these methods directly hallucinate pixels unlike the flow-based FRC methods, the predicted frames tend to be blurry when fast-moving objects are present.

Instruments

Tool / Program Availability Max. frame increase multiplier
AviSynth MSU Frame Rate Conversion Filter commercial any positive integer number
Adobe Premiere Pro commercial 100
Vegas Pro commercial 100
Topaz Video Enhance AI commercial 100
Advanced Frame Rate Converter (AFRC) free any positive integer number
FlowFrames Video AI Interpolation Free Up to 8x
AviSynth MSU Frame Rate Conversion Filter
The AviSynth MSU Frame Rate Conversion Filter is an open-source tool intended for video frame rate up-conversion. It increases the frame rate by an integer factor. It allows, for example, to convert a video with 15 fps into a video with 30 fps.
Adobe Premiere Pro
Adobe Premiere Pro is a commercial video editing software program that allows you to slow down your video using optical flow and time remapping effects to conventionally shot footage to create better looking and smoother slow motion.
Vegas Pro
Vegas Pro also is a commercial video editing software program. There is a method to make slow motion video too. To perform it you need to choose the motion magnitude in your video and percentages of playback speed.
Topaz Video Enhance AI
Topaz Video Enhance AI has the Chronos AI model which uses deep learning to increase video frame rate without artifacts. This algorithm generates new frames that are often indistinguishable from frames captured in-camera.
Advanced Frame Rate Converter (AFRC)
Main advantage of AFRC algorithm is using of several quality enhancement techniques such as adaptive artifact masking, black stripe processing and occlusion tracking:
  • adaptive artifact masking technique allows to make artifacts less noticeable for eyes thus increasing the integral quality of processed video;
  • black stripe processing allows to avoid artifacts which are commonly appeared in interpolated frames in case of black stripe presented near frame edges;
  • occlusion tracking performs high quality restoration of interpolated frames near edges in case of presence of motion with direction to/from the frame edge.

See also

References

  1. ↑ Tamasi, Tony (3 December 2019). "What is Frame Rate and Why is it Important to PC Gaming?". https://www.nvidia.com/en-us/geforce/news/what-is-fps-and-how-it-helps-you-win-games/. 
  2. ↑ 2.0 2.1 Read, Paul; Meyer, Mark-Paul; Gamma Group (2000). Restoration of motion picture film. Conservation and Museology. Butterworth-Heinemann. pp. 24–26. ISBN 978-0-7506-2793-1. https://books.google.com/books?id=jzbUUL0xJAEC&pg=PA24. 
  3. ↑ James Davis (1986), "Humans perceive flicker artefacts at 500 Hz", Sci. Rep. 5: 7861, doi:10.1038/srep07861, PMID 25644611 
  4. ↑ Potter, Mary C. (December 28, 2013). "Detecting meaning in RSVP at 13 ms per picture". Attention, Perception, & Psychophysics 76 (2): 270–279. doi:10.3758/s13414-013-0605-z. PMID 24374558. https://dspace.mit.edu/bitstream/1721.1/107157/1/13414_2013_605_ReferencePDF.pdf. 
  5. ↑ Robert Efron (1973). "Conservation of temporal information by perceptual systems". Perception & Psychophysics 14 (3): 518–530. doi:10.3758/bf03211193. 
  6. ↑ Brown, Julie (2014). "Audio-visual Palimpsests: Resynchronizing Silent Films with 'Special' Music". in David Neumeyer. The Oxford Handbook of Film Music Studies. Oxford University Press. p. 588. ISBN 978-0195328493. https://books.google.com/books?id=sWdBAQAAQBAJ&pg=PA588. 
  7. ↑ Kerr, Walter (1975). Silent Clowns. Knopf. p. 36. ISBN 978-0394469072. https://archive.org/details/silentclowns00kerrrich. 
  8. ↑ Card, James (1994). Seductive cinema: the art of silent film. Knopf. p. 53. ISBN 978-0394572185. https://archive.org/details/seductivecinemaa00card/page/53. 
  9. ↑ 9.0 9.1 9.2 Brownlow, Kevin (Summer 1980). "Silent Films: What Was the Right Speed?". Sight & Sound 49 (3): 164–167. http://www.cinemaweb.com/silentfilm/bookshelf/18_kb_2.htm. Retrieved 2 May 2012. 
  10. ↑ Elsaesser, Thomas; Barker, Adam (1990). Early cinema: space, frame, narrative. BFI Publishing. p. 284. ISBN 978-0-85170-244-5. 
  11. ↑ TWiT Netcast Network (2017-03-30), How 24 FPS Became Standard, https://www.youtube.com/watch?v=UcjYqfMaeHU, retrieved 2017-03-31 
  12. ↑ Chew, Johnny. "What Are Ones, Twos, and Threes in Animation?". Lifewire. https://www.lifewire.com/what-are-ones-twos-and-threes-4057778. 
  13. ↑ Whitaker, Harold; Sito, John Halas; updated by Tim (2009). Timing for animation (2nd ed.). Amsterdam: Elsevier/Focal Press. p. 52. ISBN 978-0240521602. https://books.google.com/books?id=yMgqBgAAQBAJ&pg=PA52. Retrieved August 8, 2018. 
  14. ↑ "Shot on threes (ones, twos, etc.)". https://www.animenewsnetwork.com/encyclopedia/lexicon.php?id=61. 
  15. ↑ CLIP STUDIO (12 February 2016). "CLIP STUDIO PAINT アニメーション機能の使い方". https://www.youtube.com/watch?v=qQalCLD6ky0&t=833. 
  16. ↑ High Frame-Rate Television, BBC White Paper WHP 169, September 2008, M. Armstrong, D. Flynn, M. Hammond, PAWAN Jahajpuria S. Jolly, R. Salmon.
  17. ↑ Jon Fingas (November 27, 2014), "James Cameron's 'Avatar' sequels will stick to 48 frames per second", Engadget, https://www.engadget.com/2014/11/27/avatar-sequels-to-shoot-at-48fps/, retrieved April 15, 2017 
  18. ↑ Whaley, Sean (21 November 2018). "What is Frame Rate and Why is it Important to PC Gaming?". https://www.hp.com/us-en/shop/tech-takes/what-is-frame-rate. 
  19. ↑ Simon, Niklaus; Long, Mai; Feng, Liu (2017). "Video frame interpolation via adaptive separable convolution". ICCV. 
  20. ↑ Huaizu, Jiang; Deqing, Sun; Varun, Jampani; Ming-Hsuan, Yang; Erik, Learned-Miller; Jan, Kautz (2018). "Super slomo: High quality estimation of multiple intermediate frames for video interpolation". ICCV. 
  21. ↑ Shurui, Gui; Chaoyue, Wang; Qihua, Chen; Dacheng, Tao (2020). "2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)". IEEE. pp. 14001–14010. doi:10.1109/CVPR42600.2020.01402. ISBN 978-1-7281-7169-2. 
  22. ↑ Myungsub, Choi; Heewon, Kim; Bohyung, Han; Ning, Xu; Kyoung, Mu Lee (2020). "Channel Attention is All You Need for Video Frame Interpolation". Proceedings of the AAAI Conference on Artificial Intelligence (AAAI) 34 (7): 10663–10671. doi:10.1609/aaai.v34i07.6693. 

External links

(Wayback Machine copy)

{{Navbox |name = Video processing |title = Video processing |state = autocollapse |listclass = hlist |basestyle = text-align:center;

|group1 = Post-processing |list1 =

|group2 = Special processing |list2 =

|group3 = 24 to 30 fps conversion |list3 =

|group4 = 30 to 24 fps conversion |list4 =

}}



Retrieved from "https://handwiki.org/wiki/index.php?title=Frame_rate&oldid=3371136"