Parallel IO

From HandWiki

Parallel I/O, in the context of a computer, means the performance of multiple input/output operations at the same time, for instance simultaneously outputs to storage devices and display devices.[1] It is a fundamental feature of operating systems.[2]

One particular instance is parallel writing of data to disk; when file data is spread across multiple disks, for example in a RAID array, one can store multiple parts of the data at the same time, thereby achieving higher write speeds than with a single device.[3][4]

Other ways of parallel access to data include: Parallel Virtual File System, Lustre, GFS etc.

Features

Scientific computing

It is used for scientific computing and not for databases. It breaks up support into multiple layers including High level I/O library, Middleware layer and Parallel file system.[5] Parallel File System manages the single view, maintains logical space and provides access to data files.[6]

Storage

A single file may be stripped across one or more object storage target, which increases the bandwidth while accessing the file and available disk space.[7] The caches are larger in Parallel I/O and shared through distributed memory systems.[8][9][10][11]

Breakthroughs

Companies have been running Parallel I/O on their servers to achieve results with regard to price and performance. Parallel processing is especially critical for scientific calculations where applications are not only CPU but also are I/O bound.[12]

See also

References

  1. "Parallel I/O". Johns Hopkins University. http://hssl.cs.jhu.edu/~randal/419/lectures/L15.ParallelIO.pdf. 
  2. "Introduction to Parallel I/O". Oak Ridge National Laboratory. https://www.olcf.ornl.gov/wp-content/uploads/2011/10/Fall_IO.pdf. 
  3. "Introduction: The Parallel I/O Stack". Cornell University. http://www.cac.cornell.edu/education/training/ParallelMay2012/ParallelIOMay2012.pdf. 
  4. "Introduction to Parallel I/O". The University of Texas at Austin. https://www.tacc.utexas.edu/documents/13601/900558/MPI-IO-Final.pdf/eea9d7d3-4b81-471c-b244-41498070e35d. 
  5. "Parallel I/O". Scientific Computing Department. http://www.scd.stfc.ac.uk//support/44958.aspx. 
  6. "A Comprehensive Look at High Performance Parallel I/O". Berkeley Lab. http://cs.lbl.gov/news-media/news/2014/a-comprehensive-look-at-high-performance-parallel-i-o/. 
  7. http://calcul.math.cnrs.fr/Documents/Manifestations/CIRA2011/2011-01_haefele_parallel_IO-workshop_Lyon.pdf
  8. https://www.olcf.ornl.gov/wp-content/uploads/2013/05/OLCF-Data-Intro-IO-Gerber-FINAL.pdf
  9. "A Comprehensive Look at High Performance Parallel I/O". http://cs.lbl.gov/news-media/news/2014/a-comprehensive-look-at-high-performance-parallel-i-o/. 
  10. "Parallel I/O – Why, How, and Where to?". 2015-04-09. https://hdfgroup.org/wp/2015/04/parallel-io-why-how-and-where-to-hdf5/. 
  11. Teng Wang; Kevin Vasko; Zhuo Liu; Hui Chen; Weikuan Yu (2016). "Enhance parallel input/output with cross-bundle aggregation". The International Journal of High Performance Computing Applications 30 (2): 241–256. doi:10.1177/1094342015618017. 
  12. Laghave, Nikhil; Sosonkina, Masha; Maris, Pieter; Vary, James P. (2009-05-25). Benefits of Parallel I/O in Ab Initio Nuclear Physics Calculations. 5544. 84–93. doi:10.1007/978-3-642-01970-8_9. ISBN 9783642019692.