Software:Kubeflow
Original author(s) | |
---|---|
Developer(s) | Kubeflow Contributors[1] - AWS, Bloomberg, Google, IBM, NVIDIA, Nutanix, Red Hat, Arrikto, and others |
Initial release | April 5, 2018[2] |
Stable release | 1.8[3]
/ November 1, 2023 |
Repository | github |
Written in | Go, Python |
Platform | Kubernetes |
Type | Machine Learning Platform |
License | Apache License 2.0 |
Website | kubeflow |
Kubeflow is an open-source platform for machine learning and MLOps on Kubernetes introduced by Google. The different stages in a typical machine learning lifecycle are represented with different software components in Kubeflow, including model development (Kubeflow Notebooks[4]), model training (Kubeflow Pipelines,[5] Kubeflow Training Operator[6]), model serving (KServe[lower-alpha 1][7]), and automated machine learning (Katib[8]).
Each component of Kubeflow can be deployed separately, and it is not a requirement to deploy every component.[9]
History
The Kubeflow project was first announced at KubeCon + CloudNativeCon North America 2017 by Google engineers David Aronchick, Jeremy Lewi, and Vishnu Kannan[10] to address a perceived lack of flexible options for building production-ready machine learning systems.[11] The project has also stated it began as a way for Google to open-source how they ran TensorFlow internally.[12]
The first release of Kubeflow (Kubeflow 0.1) was announced at KubeCon + CloudNativeCon Europe 2018[13].[14] Kubeflow 1.0 was released in March 2020 via a public blog post announcing that many Kubeflow components were graduating to a "stable status", indicating they were now ready for production usage.[15]
In October 2022, Google announced that the Kubeflow project had applied to join the Cloud Native Computing Foundation.[16][17] In July 2023, the foundation voted to accept Kubeflow as an incubating stage project.[18][19]
Components
Kubeflow Notebooks for model development
Machine learning models are developed in the notebooks component called Kubeflow Notebooks. The component runs web-based development environments inside a Kubernetes cluster, with native support for Jupyter Notebook, Visual Studio Code, and RStudio.[20]
Kubeflow Pipelines for model training
Once developed, models are trained in the Kubeflow Pipelines component. The component acts as a platform for building and deploying portable, scalable machine learning workflows based on Docker containers.[21] Google Cloud Platform has adopted the Kubeflow Pipelines DSL within its Vertex AI Pipelines product.[22]
Kubeflow Training Operator for model training
For certain machine learning models and libraries, the Kubeflow Training Operator component provides Kubernetes custom resources support. The component runs distributed or non-distributed TensorFlow, PyTorch, Apache MXNet, XGBoost, and MPI training jobs on Kubernetes.[6]
KServe for model serving
The KServe component (previously named KFServing[23]) provides Kubernetes custom resources for serving machine learning models on arbitrary frameworks including TensorFlow, XGBoost, scikit-learn, PyTorch, and ONNX.[24] KServe was developed collaboratively by Google, IBM, Bloomberg, NVIDIA, and Seldon.[23] Publicly disclosed adopters of KServe include Bloomberg,[25] Gojek,[26] and others.[27]
Katib for automated machine learning
Lastly, Kubeflow includes a component for automated training and development of machine learning models, the Katib component. It is described as a Kubernetes-native project and features hyperparameter tuning, early stopping, and neural architecture search.[28]
Release timeline
Notes
References
- ↑ "Kubeflow Website - Working Groups" (in en). https://kubeflow.org/docs/about/community/#kubeflow-working-groups.
- ↑ 2.0 2.1 "Kubeflow 0.1 - Release Tag" (in en). https://github.com/kubeflow/kubeflow/releases/tag/v0.1.0/.
- ↑ 3.0 3.1 "Kubeflow 1.8 - Release Information" (in en). https://www.kubeflow.org/docs/releases/kubeflow-1.8/.
- ↑ "Kubeflow Website - Kubeflow Notebooks" (in en). https://www.kubeflow.org/docs/components/notebooks/.
- ↑ "Kubeflow Website - Kubeflow Pipelines" (in en). https://www.kubeflow.org/docs/components/pipelines/.
- ↑ 6.0 6.1 "Kubeflow GitHub - Kubeflow Training Operator" (in en). https://github.com/kubeflow/training-operator.
- ↑ "Kubeflow Website - KServe" (in en). https://www.kubeflow.org/docs/external-add-ons/kserve/.
- ↑ "Kubeflow Website - Katib" (in en). https://www.kubeflow.org/docs/components/katib/.
- ↑ "Kubeflow Website - Installing Kubeflow" (in en). https://www.kubeflow.org/docs/started/installing-kubeflow/.
- ↑ ""Hot Dogs or Not" - At Scale with Kubernetes [I - Vish Kannan & David Aronchick, Google"] (in en). https://www.youtube.com/watch?v=R3dVF5wWz-g.
- ↑ "Introducing Kubeflow - A Composable, Portable, Scalable ML Stack Built for Kubernetes" (in en). 21 December 2017. https://kubernetes.io/blog/2017/12/introducing-kubeflow-composable/.
- ↑ "Kubeflow Website - History" (in en). https://www.kubeflow.org/docs/started/introduction/#history.
- ↑ "Google-led Kubeflow, machine learning for Kubernetes, begins to take shape" (in en). 4 May 2018. https://techcrunch.com/2018/05/04/google-kubeflow-machine-learning-for-kubernetes-begins-to-take-shape/.
- ↑ "Announcing Kubeflow 0.1" (in en). 4 May 2018. https://kubernetes.io/blog/2018/05/04/announcing-kubeflow-0.1/.
- ↑ "Kubeflow 1.0: Cloud-Native ML for Everyone" (in en). 2 March 2020. https://blog.kubeflow.org/releases/2020/03/02/kubeflow-1-0-cloud-native-ml-for-everyone.html.
- ↑ Lamkin, Thea (2022-10-24). "Kubeflow has applied to become a CNCF incubating project" (in en). https://blog.kubeflow.org/kubeflow-applied-cncf-incubating/.
- ↑ "Kubeflow applies to become a CNCF incubating project". 2022-10-24. https://opensource.googleblog.com/2022/10/kubeflow-applies-to-become-a-cncf-incubating-project.html.
- ↑ "Kubeflow brings MLOps to the CNCF Incubator" (in en-US). 2023-07-25. https://www.cncf.io/blog/2023/07/25/kubeflow-brings-mlops-to-the-cncf-incubator/.
- ↑ "Kubeflow joins the CNCF family". 2023-07-25. https://opensource.googleblog.com/2023/07/kubeflow-joins-cncf-family.html.
- ↑ "Kubeflow Website - Kubeflow Notebooks Overview" (in en). https://www.kubeflow.org/docs/components/notebooks/overview/.
- ↑ "Kubeflow Website - Kubeflow Pipelines Introduction" (in en). https://www.kubeflow.org/docs/components/pipelines/introduction/.
- ↑ "Vertex AI - Building a pipeline" (in en). https://cloud.google.com/vertex-ai/docs/pipelines/build-pipeline.
- ↑ 23.0 23.1 23.2 "KServe: The next generation of KFServing" (in en). 27 September 2021. https://blog.kubeflow.org/release/official/2021/09/27/kfserving-transition.
- ↑ "KServe GitHub" (in en). https://github.com/kserve/kserve.
- ↑ "The journey to build Bloomberg's ML Inference Platform Using KServe (formerly KFServing)" (in en). Bloomberg L.p.. 12 October 2021. https://www.bloomberg.com/company/stories/the-journey-to-build-bloombergs-ml-inference-platform-using-kserve-formerly-kfserving/.
- ↑ "Merlin: Making ML Model Deployments Magical" (in en). https://www.gojek.io/blog/merlin-making-ml-model-deployments-magical.
- ↑ "KServe Website - Adopters of KServe" (in en). https://kserve.github.io/website/master/community/adopters/.
- ↑ "Kubeflow GitHub - Katib" (in en). https://github.com/kubeflow/katib.
- ↑ "Kubeflow 0.2 - Release Tag" (in en). https://github.com/kubeflow/kubeflow/releases/tag/v0.2.0/.
- ↑ "Kubeflow 0.3 - Release Tag" (in en). https://github.com/kubeflow/kubeflow/releases/tag/v0.3.0/.
- ↑ "Kubeflow 0.4 - Release Tag" (in en). https://github.com/kubeflow/kubeflow/releases/tag/v0.4.0/.
- ↑ "Kubeflow 0.5 - Release Tag" (in en). https://github.com/kubeflow/kubeflow/releases/tag/v0.5.0/.
- ↑ "Kubeflow 0.6 - Release Information" (in en). https://www.kubeflow.org/docs/releases/kubeflow-0.6/.
- ↑ "Kubeflow 0.7 - Release Information" (in en). https://www.kubeflow.org/docs/releases/kubeflow-0.7/.
- ↑ "Kubeflow 1.0 - Release Information" (in en). https://www.kubeflow.org/docs/releases/kubeflow-1.0/.
- ↑ "Kubeflow 1.1 - Release Information" (in en). https://www.kubeflow.org/docs/releases/kubeflow-1.1/.
- ↑ "Kubeflow 1.2 - Release Information" (in en). https://www.kubeflow.org/docs/releases/kubeflow-1.2/.
- ↑ "Kubeflow 1.3 - Release Information" (in en). https://www.kubeflow.org/docs/releases/kubeflow-1.3/.
- ↑ "Kubeflow 1.4 - Release Information" (in en). https://www.kubeflow.org/docs/releases/kubeflow-1.4/.
- ↑ "Kubeflow 1.5 - Release Information" (in en). https://www.kubeflow.org/docs/releases/kubeflow-1.5/.
- ↑ "Kubeflow 1.6 - Release Information" (in en). https://www.kubeflow.org/docs/releases/kubeflow-1.6/.
- ↑ "Kubeflow 1.7 - Release Information" (in en). https://www.kubeflow.org/docs/releases/kubeflow-1.7/.
External links
Original source: https://en.wikipedia.org/wiki/Kubeflow.
Read more |