Software:Google Cloud Dataproc
 Google Cloud Dataproc | |
| Original author(s) | Google Cloud Platform |
|---|---|
| Developer(s) | Google Cloud Platform |
| Initial release | February 22, 2016 |
| Stable release | 1.2.31
/ April 13, 2018 |
| Operating system | |
| Available in | English |
| Type | Managed Spark and Hadoop |
| Website | cloud |
Google Cloud Dataproc (Cloud Dataproc) is a cloud-based managed Spark and Hadoop service offered on Google Cloud Platform. Cloud Dataproc utilizes many Google Cloud Platform technologies such as Google Compute Engine and Google Cloud Storage to offer fully managed clusters running popular[1][2] data processing frameworks such as Apache Hadoop and Apache Spark.[3]
Design
Cloud Dataproc is a Platform as a service (PaaS) product designed to combine the Spark and Hadoop frameworks with many common cloud computing patterns. Cloud Dataproc separates compute and storage, which is a relatively common design for many cloud Hadoop offerings. Cloud Dataproc utilizes Google Compute Engine virtual machines for compute and Google Cloud Storage for file storage. Cloud Dataproc has a set of control and integration mechanisms that coordinate the lifecycle, management, and coordination of clusters. Cloud Dataproc is integrated with the YARN application manager to make managing and using clusters easier.[4]
Cloud Dataproc includes many open source packages used for data processing, including items from the Spark and Hadoop ecosystem, and open source tools to connect these frameworks with other Google Cloud Platform products.[5]
History
Cloud Dataproc was released as a publicly available beta service on September 23, 2015[6] and entered public general availability on February 22, 2016.[7]
Similar products
See also
External links
- Cloud Dataproc release notes
- Google Cloud Platform site
- Cloud Dataproc questions on Stack Overflow
- Cloud Dataproc discussion list
References
- ↑ "Fortune Spark Survey". http://fortune.com/2015/09/25/apache-spark-survey/.
- ↑ "TechCrunch - On The Growth Of Apache Spark". https://techcrunch.com/2015/03/19/on-the-growth-of-apache-spark/.
- ↑ "Official Google Cloud Dataproc page". https://cloud.google.com/dataproc/.
- ↑ "Cloud Dataproc FAQ". https://cloud.google.com/dataproc/faq#how_does_cloud_dataproc_work.
- ↑ "Cloud Dataproc - Versions". https://cloud.google.com/dataproc/dataproc-versions.
- ↑ "Cloud Dataproc Beta Blog Post". https://cloudplatform.googleblog.com/2015/09/Google-Cloud-Dataproc-Making-Spark-and-Hadoop-Easier-Faster-and-Cheaper.html.
- ↑ "Cloud Dataproc GA blog post". https://cloudplatform.googleblog.com/2016/02/Google-Cloud-Dataproc-managed-Spark-and-Hadoop-service-now-GA.html.
 |
