Vision processing unit: Difference between revisions

From HandWiki
imported>JMinHep
fixing
 
correction
 
Line 1: Line 1:
{{short description|Emerging class of microprocessor}}
{{short description|Emerging class of microprocessor}}
A '''vision processing unit''' ('''VPU''') is (as of 2023) an emerging class of [[Engineering:Microprocessor|microprocessor]]; it is a specific type of [[AI accelerator]], designed to [[Hardware acceleration|accelerate]] [[Machine vision|machine vision]] tasks.<ref>{{cite web|title= A third type of processor for AR/VR: Movidius' Myriad 2 VPU|url=http://www.tomshardware.com/news/movidiud-myriad2-vpu-vision-processing-vr,30850.html|author=Seth Colaner|author2=Matthew Humrick|date=January 3, 2016|work=Tom's Hardware}}</ref><ref>{{cite web|title=The rise of VPUs: Giving Eyes to Machines|url=http://www.digit.in/general/the-rise-of-vpus-giving-eyes-to-machines-29561.html|work=Digit.in|author=Prasid Banerje|date=March 28, 2016}}</ref>
A '''vision processing unit''' ('''VPU''') is (as of 2023) an emerging class of [[Engineering:Microprocessor|microprocessor]]; it is a specific type of [[AI accelerator]], designed to [[Hardware acceleration|accelerate]] [[Machine vision|machine vision]] tasks.<ref>{{cite web|title= A third type of processor for AR/VR: Movidius' Myriad 2 VPU|url=http://www.tomshardware.com/news/movidiud-myriad2-vpu-vision-processing-vr,30850.html|author=Seth Colaner|author2=Matthew Humrick|date=January 3, 2016|work=Tom's Hardware}}</ref><ref>{{cite web|title=The rise of VPUs: Giving Eyes to Machines|url=http://www.digit.in/general/the-rise-of-vpus-giving-eyes-to-machines-29561.html|work=Digit.in|author=Prasid Banerje|date=March 28, 2016|access-date=April 18, 2016|archive-date=September 2, 2017|archive-url=https://web.archive.org/web/20170902012608/http://www.digit.in/general/the-rise-of-vpus-giving-eyes-to-machines-29561.html|url-status=dead}}</ref>
<!--- see List of AI accelerators for proposed article AI accelerator if a consensus is reached, this article can be re-worded in the context of shared information there--->
<!--- see List of AI accelerators for proposed article AI accelerator if a consensus is reached, this article can be re-worded in the context of shared information there--->


== Overview ==
== Overview ==
Vision processing units are distinct from video processing units (which are specialised for [[Video codec|video encoding and decoding]]) in their suitability for running [[Machine vision|machine vision algorithms]] such as CNN ([[Convolutional neural network|convolutional neural network]]s), SIFT ([[Scale-invariant feature transform|scale-invariant feature transform]]) and similar.
Vision processing units are distinct from [[Graphics processing unit|graphics processing unit]]s (which are specialised for [[Video codec|video encoding and decoding]]) in their suitability for running [[Machine vision|machine vision algorithms]] such as CNN ([[Convolutional neural network|convolutional neural network]]s) and SIFT ([[Scale-invariant feature transform|scale-invariant feature transform]]).


They may include [[Interface (computing)|direct interfaces]] to take data from cameras (bypassing any off chip buffers), and have a greater emphasis on on-chip [[Dataflow|dataflow]] between many parallel execution units with [[Scratchpad memory|scratchpad memory]], like a manycore [[Engineering:Digital signal processor|DSP]]. But, like video processing units, they may have a focus on low precision fixed point arithmetic for [[Image processing|image processing]].
They may include [[Interface (computing)|direct interfaces]] to take data from cameras (bypassing any off chip buffers), and have a greater emphasis on on-chip [[Dataflow|dataflow]] between many parallel execution units with [[Scratchpad memory|scratchpad memory]], like a [[Spatial architecture|spatial architecture]] or a [[Manycore processor|manycore]] [[Engineering:Digital signal processor|DSP]]. But, like video processing units, they may have a focus on low precision fixed point arithmetic for [[Image processing|image processing]].


== Contrast with GPUs ==
== Contrast with GPUs ==
They are distinct from GPUs, which contain specialised hardware for rasterization and [[Texture mapping|texture mapping]] (for 3D graphics), and whose [[Memory architecture|memory architecture]] is optimised for manipulating bitmap images in off-chip memory (reading textures, and modifying frame buffers, with [[Locality of reference|random access patterns]]). VPUs are optimized for performance per watt, while GPUs mainly focus on absolute performance.
They are distinct from GPUs, which contain specialised hardware for rasterization and [[Texture mapping|texture mapping]] (for 3D graphics), and whose [[Memory architecture|memory architecture]] is optimised for manipulating bitmap images in off-chip memory (reading textures, and modifying frame buffers, with [[Locality of reference|random access patterns]]). VPUs are optimized for performance per watt, while GPUs mainly focus on absolute performance.


Target markets are robotics, the [[Internet of things|internet of things]] (IoT), new classes of digital cameras for [[Virtual reality|virtual reality]] and [[Augmented reality|augmented reality]], [[Smart camera|smart camera]]s, and integrating machine vision acceleration into [[Engineering:Smartphone|smartphone]]s and other mobile devices.
Target markets are [[Engineering:Robotics|robotics]], the [[Internet of things|internet of things]] (IoT), new classes of digital cameras for [[Virtual reality|virtual reality]] and [[Augmented reality|augmented reality]], [[Smart camera|smart camera]]s, and integrating machine vision acceleration into [[Engineering:Smartphone|smartphone]]s and other mobile devices.


== Examples ==
== Examples ==
* Movidius Myriad X, which is the third-generation vision processing unit in the Myriad VPU line from [[Company:Intel|Intel Corporation]].<ref>{{Cite web|url=https://www.intel.com/content/www/us/en/products/details/processors/movidius-vpu.html|title=Intel® Movidius™ Vision Processing Units (VPUs)|website=Intel}}</ref>
* Movidius Myriad X, which is the third-generation vision processing unit in the Myriad VPU line from [[Company:Intel|Intel Corporation]].<ref>{{Cite web|url=https://www.intel.com/content/www/us/en/products/details/processors/movidius-vpu.html|title=Intel® Movidius™ Vision Processing Units (VPUs)|website=Intel}}</ref>
* [[Engineering:Movidius Myriad 2|Movidius Myriad 2]], which finds use in Google Project Tango,<ref name="RiseOfVPUs">{{cite web|last1=Weckler|first1=Adrian|title=Dublin tech firm Movidius to power Google's new virtual reality headset|url=http://www.independent.ie/business/technology/news/dublin-tech-firm-movidius-to-power-googles-new-virtual-reality-headset-34449883.html|website=Independent.ie|access-date=15 March 2016}}</ref> [[Engineering:Google Clips|Google Clips]] and DJI drones<ref>{{cite web|url=https://www.movidius.com/news/dji-brings-two-new-flagship-drones-to-lineup-featuring-myriad-2-vpus|title=DJI Brings Two New Flagship Drones to Lineup Featuring Myriad 2 VPUs - Machine Vision Technology - Movidius|website=www.movidius.com}}</ref>
* [[Engineering:Movidius Myriad 2|Movidius Myriad 2]], which finds use in Google Project Tango,<ref name="RiseOfVPUs">{{cite web|last1=Weckler|first1=Adrian|title=Dublin tech firm Movidius to power Google's new virtual reality headset|url=http://www.independent.ie/business/technology/news/dublin-tech-firm-movidius-to-power-googles-new-virtual-reality-headset-34449883.html|website=Independent.ie|date=14 February 2016 |access-date=15 March 2016}}</ref> [[Engineering:Google Clips|Google Clips]] and DJI drones<ref>{{cite web|url=https://www.movidius.com/news/dji-brings-two-new-flagship-drones-to-lineup-featuring-myriad-2-vpus|title=DJI Brings Two New Flagship Drones to Lineup Featuring Myriad 2 VPUs - Machine Vision Technology - Movidius|website=www.movidius.com}}</ref>
*[[Engineering:Pixel Visual Core|Pixel Visual Core]] (PVC), which is a fully programmable [[Engineering:Image processor|Image]], Vision and [[AI accelerator|AI]] processor for mobile devices
*[[Engineering:Pixel Visual Core|Pixel Visual Core]] (PVC), which is a fully programmable [[Engineering:Image processor|Image]], Vision and [[AI accelerator|AI]] processor for mobile devices
* [[Engineering:Microsoft HoloLens|Microsoft HoloLens]], which includes an accelerator referred to as a ''Holographic Processing Unit'' (complementary to its CPU and GPU), aimed at interpreting camera inputs, to accelerate environment tracking & vision for augmented reality applications.<ref>{{cite web|url=http://www.pcworld.com/article/2917512/microsoft-designed-a-special-processor-to-handle-hololens-data.html|title=Microsoft dives deeper into HoloLens details: 'Holographic processor' role revealed|date=May 1, 2015|author=Fred O'Connor|work=PCWorld}}</ref>
* [[Engineering:Microsoft HoloLens|Microsoft HoloLens]], which includes an accelerator referred to as a ''holographic processing unit'' (complementary to its CPU and GPU), aimed at interpreting camera inputs, to accelerate environment tracking and vision for augmented reality applications.<ref>{{cite web|url=http://www.pcworld.com/article/2917512/microsoft-designed-a-special-processor-to-handle-hololens-data.html|title=Microsoft dives deeper into HoloLens details: 'Holographic processor' role revealed|date=May 1, 2015|author=Fred O'Connor|work=PCWorld}}</ref>
* Eyeriss, a design from MIT intended for running [[Convolutional neural network|convolutional neural network]]s.<ref>{{cite web|url=https://www.mit.edu/~sze/eyeriss.html|title=Eyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks|author=Chen, Yu-Hsin|author2=Krishna, Tushar|author3=Emer, Joel|author4=Sze, Vivienne|author4-link=Vivienne Sze|name-list-style=amp|work=IEEE International Solid-State Circuits Conference, ISSCC 2016, Digest of Technical Papers|year=2016|pages=262–263}}</ref>
* Eyeriss, a [[Spatial architecture|spatial architecture]] designed from MIT intended for running [[Convolutional neural network|convolutional neural network]]s.<ref>{{cite web|url=https://www.mit.edu/~sze/eyeriss.html|title=Eyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks|author=Chen, Yu-Hsin|author2=Krishna, Tushar|author3=Emer, Joel|author4=Sze, Vivienne|author4-link=Vivienne Sze|name-list-style=amp|work=IEEE International Solid-State Circuits Conference, ISSCC 2016, Digest of Technical Papers|year=2016|pages=262–263}}</ref>
* NeuFlow, a design by [[Biography:Yann LeCun|Yann LeCun]] (implemented in FPGA) for accelerating convolutions, using a dataflow architecture.
* NeuFlow, a design by [[Biography:Yann LeCun|Yann LeCun]] (implemented in FPGA) for accelerating convolutions, using a dataflow architecture.
* Mobileye EyeQ, by [[Company:Mobileye|Mobileye]]
* Mobileye EyeQ, by [[Company:Mobileye|Mobileye]]
Line 30: Line 30:
* [[Company:IBM|IBM]] [[Engineering:TrueNorth|TrueNorth]], a neuromorphic processor aimed at similar sensor data [[Pattern recognition|pattern recognition]] and intelligence tasks, including video/audio.
* [[Company:IBM|IBM]] [[Engineering:TrueNorth|TrueNorth]], a neuromorphic processor aimed at similar sensor data [[Pattern recognition|pattern recognition]] and intelligence tasks, including video/audio.
* Qualcomm Zeroth Neural processing unit, another entry in the emerging class of sensor/AI oriented chips.<ref>{{cite web|title=Introducing Qualcomm Zeroth Processors: Brain-Inspired Computing|url=https://www.qualcomm.com/news/onq/2013/10/10/introducing-qualcomm-zeroth-processors-brain-inspired-computing|date=October 10, 2013|work=Qualcomm}}</ref>
* Qualcomm Zeroth Neural processing unit, another entry in the emerging class of sensor/AI oriented chips.<ref>{{cite web|title=Introducing Qualcomm Zeroth Processors: Brain-Inspired Computing|url=https://www.qualcomm.com/news/onq/2013/10/10/introducing-qualcomm-zeroth-processors-brain-inspired-computing|date=October 10, 2013|work=Qualcomm}}</ref>
* All models of Intel [[Engineering:Meteor Lake|Meteor Lake]] processors have a Versatile Processor Unit (VPU) built-in for accelerating [[Statistical inference|inference]] for computer vision and deep learning.<ref>{{Cite web|url=https://www.pcmag.com/news/intel-to-bring-a-vpu-processor-unit-to-14th-gen-meteor-lake-chips|title=Intel to Bring a 'VPU' Processor Unit to 14th Gen Meteor Lake Chips|website=PCMAG}}</ref>
* All models of Intel [[Engineering:Meteor Lake|Meteor Lake]] processors have a Versatile Processor Unit (VPU) built-in for accelerating [[Statistical inference|inference]] for computer vision and deep learning.<ref>{{Cite web|url=https://www.pcmag.com/news/intel-to-bring-a-vpu-processor-unit-to-14th-gen-meteor-lake-chips|title=Intel to Bring a 'VPU' Processor Unit to 14th Gen Meteor Lake Chips|website=PCMAG|date=August 2022 }}</ref>


== See also ==
== See also ==
Line 54: Line 54:


[[Category:Microprocessors]]
[[Category:Microprocessors]]
[[Category:AI accelerators]]
[[Category:Machine vision]]
[[Category:Machine vision]]


{{Sourceattribution|Vision processing unit}}
{{Sourceattribution|Vision processing unit}}

Latest revision as of 02:17, 24 May 2026

Short description: Emerging class of microprocessor

A vision processing unit (VPU) is (as of 2023) an emerging class of microprocessor; it is a specific type of AI accelerator, designed to accelerate machine vision tasks.[1][2]

Overview

Vision processing units are distinct from graphics processing units (which are specialised for video encoding and decoding) in their suitability for running machine vision algorithms such as CNN (convolutional neural networks) and SIFT (scale-invariant feature transform).

They may include direct interfaces to take data from cameras (bypassing any off chip buffers), and have a greater emphasis on on-chip dataflow between many parallel execution units with scratchpad memory, like a spatial architecture or a manycore DSP. But, like video processing units, they may have a focus on low precision fixed point arithmetic for image processing.

Contrast with GPUs

They are distinct from GPUs, which contain specialised hardware for rasterization and texture mapping (for 3D graphics), and whose memory architecture is optimised for manipulating bitmap images in off-chip memory (reading textures, and modifying frame buffers, with random access patterns). VPUs are optimized for performance per watt, while GPUs mainly focus on absolute performance.

Target markets are robotics, the internet of things (IoT), new classes of digital cameras for virtual reality and augmented reality, smart cameras, and integrating machine vision acceleration into smartphones and other mobile devices.

Examples

  • Movidius Myriad X, which is the third-generation vision processing unit in the Myriad VPU line from Intel Corporation.[3]
  • Movidius Myriad 2, which finds use in Google Project Tango,[4] Google Clips and DJI drones[5]
  • Pixel Visual Core (PVC), which is a fully programmable Image, Vision and AI processor for mobile devices
  • Microsoft HoloLens, which includes an accelerator referred to as a holographic processing unit (complementary to its CPU and GPU), aimed at interpreting camera inputs, to accelerate environment tracking and vision for augmented reality applications.[6]
  • Eyeriss, a spatial architecture designed from MIT intended for running convolutional neural networks.[7]
  • NeuFlow, a design by Yann LeCun (implemented in FPGA) for accelerating convolutions, using a dataflow architecture.
  • Mobileye EyeQ, by Mobileye
  • Programmable Vision Accelerator (PVA), a 7-way VLIW Vision Processor designed by Nvidia.

Broader category

Main article: AI accelerator

Some processors are not described as VPUs, but are equally applicable to machine vision tasks. These may form a broader category of AI accelerators (to which VPUs may also belong), however as of 2016 there is no consensus on the name:

  • IBM TrueNorth, a neuromorphic processor aimed at similar sensor data pattern recognition and intelligence tasks, including video/audio.
  • Qualcomm Zeroth Neural processing unit, another entry in the emerging class of sensor/AI oriented chips.[8]
  • All models of Intel Meteor Lake processors have a Versatile Processor Unit (VPU) built-in for accelerating inference for computer vision and deep learning.[9]

See also

  • Adapteva Epiphany, a manycore processor with similar emphasis on on-chip dataflow, focussed on 32-bit floating point performance
  • CELL, a multicore processor with features fairly consistent with vision processing units (SIMD instructions & datatypes suitable for video, and on-chip DMA between scratchpad memories)
  • Coprocessor
  • Graphics processing unit, also commonly used to run vision algorithms. NVidia's Pascal architecture includes FP16 support, to provide a better precision/cost tradeoff for AI workloads
  • MPSoC
  • OpenCL
  • OpenVX
  • Physics processing unit, a past attempt to complement the CPU and GPU with a high throughput accelerator
  • Tensor Processing Unit, a chip used internally by Google for accelerating AI calculations

References

  1. ↑ Seth Colaner; Matthew Humrick (January 3, 2016). "A third type of processor for AR/VR: Movidius' Myriad 2 VPU". Tom's Hardware. http://www.tomshardware.com/news/movidiud-myriad2-vpu-vision-processing-vr,30850.html. 
  2. ↑ Prasid Banerje (March 28, 2016). "The rise of VPUs: Giving Eyes to Machines". Digit.in. http://www.digit.in/general/the-rise-of-vpus-giving-eyes-to-machines-29561.html. 
  3. ↑ "Intel® Movidius™ Vision Processing Units (VPUs)". https://www.intel.com/content/www/us/en/products/details/processors/movidius-vpu.html. 
  4. ↑ Weckler, Adrian (14 February 2016). "Dublin tech firm Movidius to power Google's new virtual reality headset". http://www.independent.ie/business/technology/news/dublin-tech-firm-movidius-to-power-googles-new-virtual-reality-headset-34449883.html. 
  5. ↑ "DJI Brings Two New Flagship Drones to Lineup Featuring Myriad 2 VPUs - Machine Vision Technology - Movidius". https://www.movidius.com/news/dji-brings-two-new-flagship-drones-to-lineup-featuring-myriad-2-vpus. 
  6. ↑ Fred O'Connor (May 1, 2015). "Microsoft dives deeper into HoloLens details: 'Holographic processor' role revealed". PCWorld. http://www.pcworld.com/article/2917512/microsoft-designed-a-special-processor-to-handle-hololens-data.html. 
  7. ↑ Chen, Yu-Hsin; Krishna, Tushar (2016). "Eyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks". IEEE International Solid-State Circuits Conference, ISSCC 2016, Digest of Technical Papers. pp. 262–263. https://www.mit.edu/~sze/eyeriss.html. 
  8. ↑ "Introducing Qualcomm Zeroth Processors: Brain-Inspired Computing". Qualcomm. October 10, 2013. https://www.qualcomm.com/news/onq/2013/10/10/introducing-qualcomm-zeroth-processors-brain-inspired-computing. 
  9. ↑ "Intel to Bring a 'VPU' Processor Unit to 14th Gen Meteor Lake Chips". August 2022. https://www.pcmag.com/news/intel-to-bring-a-vpu-processor-unit-to-14th-gen-meteor-lake-chips. 



Retrieved from "https://handwiki.org/wiki/index.php?title=Vision_processing_unit&oldid=4349497"