Software:Unity Catalog

From HandWiki
Unity Catalog
Developer(s)Unity Catalog community (LF AI & Data Foundation)
Initial releaseJune 12, 2024; 2 years ago (2024-06-12) (open source)
Stable release
0.6.0 / August 20, 2026; 51 days ago (2026-08-20)[1]
Repositorygithub.com/unitycatalog/unitycatalog
Written inJava, Python, TypeScript, Scala
Operating systemCross-platform
TypeMetadata catalog, data governance
LicenseApache License 2.0
Websitewww.unitycatalog.io

Unity Catalog is a metadata and governance catalog for data and artificial intelligence assets, released under the Apache License. It includes a unified namespace for tables, unstructured data volumes, and functions, with APIs that can be accessed with multiple compute engines. It also manages table formats including Delta Lake and Apache Iceberg.[2][3] The source code for the project is part of the LF AI & Data Foundation, an open source nonprofit software foundation.[4][5] The catalog was originally developed at Databricks, which also offers it as a managed service.[6]

Overview

Unity Catalog works by organizing data and AI assets in a hierarchical namespace consisting of catalogs, schemas, and other securable objects.[3][7][8] It manages structured tables, supporting formats such as Delta Lake, Apache Iceberg, Apache Parquet, CSV, and JSON; and unstructured data through volumes; and machine-learning assets such as functions and models.[3][9]

The governance features in the project include credential vending, a feature where the catalog server issues scoped credentials to clients accessing underlying cloud storage, and support for the Apache Iceberg REST Catalog API and Apache Hive metastore API.[3][9] By acting as a layer that exposes open APIs, implementation provides an interface for external clients, such as engines like Apache Spark to read tables, volumes, and functions that are managed by the catalog.[10]

Unity Catalog leverages an OpenAPI-based REST specification and an reference server, with client libraries and SDKs.[9][7] The server is compatible with the popular Apache Iceberg REST Catalog and provides compatibility with the Apache Hive metastore API.[3][9]

See also

References

  1. ↑ "Release v0.6.0". August 20, 2026. https://github.com/unitycatalog/unitycatalog/releases/tag/v0.6.0. 
  2. ↑ Krill, Paul (June 12, 2024). "Databricks races with Snowflake to open up data catalog source code". InfoWorld. https://www.infoworld.com/article/2337619/databricks-races-with-snowflake-to-open-up-data-catalog-source-code.html. 
  3. ↑ 3.0 3.1 3.2 3.3 3.4 "Open sourcing Unity Catalog". June 12, 2024. https://www.databricks.com/blog/open-sourcing-unity-catalog. 
  4. ↑ "Welcoming Unity Catalog to the LF AI & Data Foundation". June 20, 2024. https://lfaidata.foundation/blog/2024/06/20/welcoming-unity-catalog-to-the-lf-ai-data-foundation-a-milestone-in-open-data-and-ai-governance/. 
  5. ↑ "Unity Catalog". https://lfaidata.foundation/projects/unity-catalog/. 
  6. ↑ "What is Unity Catalog?". https://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/. 
  7. ↑ 7.0 7.1 "Unity Catalog Documentation". https://docs.unitycatalog.io/. 
  8. ↑ "Unity Catalog: Open and Universal Governance for the Lakehouse and Beyond". Proceedings of the ACM SIGMOD/PODS Conference. 2025. doi:10.1145/3722212.3724459. https://dl.acm.org/doi/abs/10.1145/3722212.3724459. 
  9. ↑ 9.0 9.1 9.2 9.3 "unitycatalog/unitycatalog". https://github.com/unitycatalog/unitycatalog. 
  10. ↑ "Databricks open-sources Unity Catalog, challenging Snowflake on interoperability for data workloads". VentureBeat. June 12, 2024. https://venturebeat.com/data-infrastructure/databricks-open-sources-unity-catalog-challenging-snowflake-on-interoperability-for-data-workloads. 



Retrieved from "https://handwiki.org/wiki/index.php?title=Software:Unity_Catalog&oldid=4625982"