Software:History and implementations of ZFS

From HandWiki
ZFS
Aktualne logo Oracle Solaris OS OSos.png
DeveloperSun Microsystems (acquired by Oracle Corporation in 2009)
Written inC, C++
OS familyUnix (System V Release 4)
Working stateCurrent
Source modelMixed open-source / closed-source
Initial releaseJune 2006; 17 years ago (2006-06) Solaris 10 6/06 ("U2")
|Final release|Latest release}}11.3[1] / October 26, 2015; 8 years ago (2015-10-26)
Marketing targetWorkstation, server
PlatformsSPARC, x86-64, IA-32 (except Solaris 11), PowerPC (Solaris 2.5.1 only)
LicenseVarious
Official websitewww.oracle.com/solaris
OpenZFS
Logo of the OpenZFS project
Initial releasePorted to various systems between 2006 and 2010. Forked from OpenSolaris August 2010; 13 years ago (2010-08)
Repositorygithub.com/openzfs/openzfs
Written inC
Operating systemOpenSolaris, illumos distributions, OpenIndiana, FreeBSD, Mac OS X Server 10.5 (only read-only support), NetBSD, Linux via third-party kernel module ("ZFS on Linux")[2] or ZFS-FUSE, OSv
Licenseopen source CDDL
Websiteopen-zfs.org

The history and implementations of ZFS covers the development of the ZFS file system.

ZFS began as part of the Sun Microsystems Solaris operating system in 2001. Large parts of Solaris - including ZFS - were published under an open source license as OpenSolaris for around 5 years from 2005, before being placed under a closed source license when Oracle Corporation acquired Sun in 2009/2010. During 2005 - 2010, the open source version of ZFS was ported to Linux, Mac OS X (continued as MacZFS) and FreeBSD. In 2010, the illumos project forked a recent version of OpenSolaris, to continue its development as an open source project, including ZFS. In 2013 the co-ordination of open source ZFS moved to an umbrella organization, OpenZFS, which allowed any person or organization that wished to use the open source version of ZFS, to collaborate in developing and maintaining a single common version of ZFS. illumos remains very closely involved with OpenZFS.

As of 2018, there are two main implementations of ZFS, both quite similar: Oracle's implementation, which is closed source and part of Solaris, and OpenZFS, which is widely used to provide ZFS on many unix-like operating systems.

Development history

Sun Microsystems (to 2010)

In 1987, AT&T Corporation and Sun announced that they were collaborating on a project to merge the most popular Unix variants on the market at that time: Berkeley Software Distribution, UNIX System V, and Xenix. This became Unix System V Release 4 (SVR4).[3] The project was released under the name Solaris, which became the successor to SunOS 4 (although SunOS 4.1.x micro releases were retroactively named Solaris 1).[4]

ZFS was designed and implemented by a team at Sun led by Jeff Bonwick, Bill Moore[5] and Matthew Ahrens. It was announced on September 14, 2004,[6] but development started in 2001.[7] Source code for ZFS was integrated into the main trunk of Solaris development on October 31, 2005,[8] and released for developers as part of build 27 of OpenSolaris on November 16, 2005. In June 2006, Sun announced that ZFS was included in the mainstream 6/06 update to Solaris 10.[9]

Historically, Solaris was developed as proprietary software. Sun Microsystems was a strong proponent of open source software. In June 2005, Sun released most of the codebase under the CDDL license, and founded the OpenSolaris open-source project.[10] Sun was an early proponent of open source software, and with OpenSolaris, Sun wanted to build a developer and user community around the software. In Solaris 10 6/06 ("U2"), Sun added the ZFS file system. During the next 5 years (2006 to 2010), Sun frequently updated ZFS with new features, and ZFS was ported to Linux, Mac OS X (continued as MacZFS) and FreeBSD, under this open source license.

The name at one point was said to stand for "Zettabyte File System",[11] but by 2006 was no longer considered to be an abbreviation.[12] A ZFS file system can store up to 256 quadrillion zettabytes (ZB).

In September 2007, NetApp sued Sun claiming that ZFS infringed some of NetApp's patents on Write Anywhere File Layout. Sun counter-sued in October the same year claiming the opposite. The lawsuits were ended in 2010 with an undisclosed settlement.[13]

Ported versions (2005 - 2010)

As the FSF claimed a CDDL and GPL legal incompatibility in 2005, Sun's implementation of the ZFS file system wasn't used as a basis for the development of a Linux kernel module, it wasn't merged into the Linux kernel mainline, and Linux distributions did not include it as a precompiled kernel module.[14][15] As a workaround, FUSE, a framework that allows file systems to run in userspace, was used on Linux as a separation layer for which the licensing issues are not in effect, although with a set of its own issues that include performance penalty.[16][17] However, the April 2016 release of Ubuntu 16.04 LTS includes CDDL-licensed ZFS on Linux[18] as a kernel module that is maintained as a separate project, outside the Linux kernel mainline, claiming license compatibility.[19][20][21]

Oracle Corporation, close source, and forking (from 2010)

In January 2010, Oracle Corporation acquired Sun Microsystems, and quickly discontinued the OpenSolaris distribution and the open source development model.[22][23] In August 2010, Oracle discontinued providing public updates to the source code of the Solaris kernel, effectively turning Solaris 11 back into a closed source proprietary operating system.[24]

In response to the changing landscape of Solaris and OpenSolaris, the illumos project was launched via webinar[25] on Thursday, 3 August 2010, as a community effort of some core Solaris engineers to continue developing the open source version of Solaris, and complete the open sourcing of those parts not already open sourced by Sun.[26] illumos was founded as a Foundation, the illumos Foundation, incorporated in the State of California as a 501(c)6 trade association. The original plan explicitly stated that illumos would not be a distribution or a fork. However, after Oracle announced discontinuing OpenSolaris, plans were made to fork the final version of the Solaris ON kernel allowing illumos to evolve into a kernel of its own.[27] As part of OpenSolaris, an open source version of ZFS was therefore integral within illumos.

ZFS was widely used within numerous platforms, as well as Solaris. Therefore, in 2013, the co-ordination of development work on the open source version of ZFS was passed to an umbrella project, OpenZFS. The OpenZFS framework allows any interested parties to collaboratively develop the core ZFS codebase in common, while individually maintaining any specific extra code which ZFS requires to function and integrate within their own systems.

ZFS releases and feature histories

Within Sun/Oracle Solaris

Within illumos and (from 2013) OpenZFS

  • 2010: illumos founded as an open source successor,[28] and continued to develop ZFS in the open. Ports of ZFS to other platforms continued porting upstream changes from illumos.[citation needed]
  • 2013: OpenZFS project founded, aiming at coordinated open-source development of ZFS. The OpenZFS project provides a common foundation for any interested groups and organizations to contribute and collaborate towards a common open source ZFS core, and in addition, to also maintain any specific code and validation processes needed for core ZFS code to work with their own individual systems.

Other ports and forks

The following is a list of open-source ZFS ports and forks:[29][30]

  • 2006: Development of a FUSE port for Linux started.
  • 2007: Apple started porting ZFS to Mac OS X.
  • 2008: A port to FreeBSD was released as part of FreeBSD 7.0.
  • 2008: Development of a native Linux port started.
  • 2009: Apple's ZFS project closed. The MacZFS project continued to develop the code.

Detailed release histories

Sun/Oracle

Legend:
Old release
Latest FOSS stable release
Latest Proprietary stable release
Latest Proprietary beta release
ZFS Filesystem Version Number Release date Significant changes
1 OpenSolaris Nevada[31] build 36 First release
2 OpenSolaris Nevada b69 Enhanced directory entries. In particular, directory entries now store the object type. For example, file, directory, named pipe, and so on, in addition to the object number.
3 OpenSolaris Nevada b77 Support for sharing ZFS file systems over SMB. Case insensitivity support. System attribute support. Integrated anti-virus support.
4 OpenSolaris Nevada b114 Properties: userquota, groupquota, userused and groupused
5 OpenSolaris Nevada b137 System attributes; symlinks now their own object type
6 Solaris 11.1 Multilevel file system support
ZFS Pool Version Number Release date Significant changes
1 OpenSolaris Nevada[31] b36 First release
2 OpenSolaris Nevada b38 Ditto Blocks
3 OpenSolaris Nevada b42 Hot spares, double-parity RAID-Z (raidz2), improved RAID-Z accounting
4 OpenSolaris Nevada b62 zpool history
5 OpenSolaris Nevada b62 gzip compression for ZFS datasets
6 OpenSolaris Nevada b62 "bootfs" pool property
7 OpenSolaris Nevada b68 ZIL: adds the capability to specify a separate Intent Log device or devices
8 OpenSolaris Nevada b69 ability to delegate zfs(1M) administrative tasks to ordinary users
9 OpenSolaris Nevada b77 CIFS server support, dataset quotas
10 OpenSolaris Nevada b77 Devices can be added to a storage pool as "cache devices"
11 OpenSolaris Nevada b94 Improved zpool scrub / resilver performance
12 OpenSolaris Nevada b96 Snapshot properties
13 OpenSolaris Nevada b98 Properties: usedbysnapshots, usedbychildren, usedbyrefreservation, and usedbydataset
14 OpenSolaris Nevada b103 passthrough-x aclinherit property support
15 OpenSolaris Nevada b114 Properties: userquota, groupquota, usuerused and groupused; also required FS v4
16 OpenSolaris Nevada b116 STMF property support
17 OpenSolaris Nevada b120 triple-parity RAID-Z
18 OpenSolaris Nevada b121 ZFS snapshot holds
19 OpenSolaris Nevada b125 ZFS log device removal
20 OpenSolaris Nevada b128 zle compression algorithm that is needed to support the ZFS deduplication properties in ZFS pool version 21, which were released concurrently
21 OpenSolaris Nevada b128 Deduplication
22 OpenSolaris Nevada b128 zfs receive properties
23 OpenSolaris Nevada b135 slim ZIL
24 OpenSolaris Nevada b137 System attributes. Symlinks now their own object type. Also requires FS v5.
25 OpenSolaris Nevada b140 Improved pool scrubbing and resilvering statistics
26 OpenSolaris Nevada b141 Improved snapshot deletion performance
27 OpenSolaris Nevada b145 Improved snapshot creation performance (particularly recursive snapshots)
28 OpenSolaris Nevada b147 Multiple virtual device replacements
29 Solaris Nevada b148 RAID-Z/mirror hybrid allocator
30 Solaris Nevada b149 ZFS encryption
31 Solaris Nevada b150 Improved 'zfs list' performance
32 Solaris Nevada b151 One MB block support
33 Solaris Nevada b163 Improved share support
34 Solaris 11.1 (0.5.11-0.175.1.0.0.24.2) Sharing with inheritance
35 Solaris 11.2 (0.5.11-0.175.2.0.0.42.0) Sequential resilver
36 Solaris 11.3 Efficient log block allocation
37 Solaris 11.3 LZ4 compression
38 Solaris 11.4 xcopy with encryption
39 Solaris 11.4 reduce resilver restart
40 Solaris 11.4 Deduplication 2
41 Solaris 11.4 Asynchronous dataset destroy
42 Solaris 11.4 Reguid: ability to change the pool guid
43 Solaris 11.4, Oracle ZFS Storage Simulator 8.7[32] RAID-Z improvements and cloud device support. Vdev remove of basic, mirror and RAID-Z[1-3] vdevs, ZFS embedded/ kernelbased servers support SMB3 and NFS 4.1
44 Solaris 11.4 Device removal

Note: The Solaris version under development by Sun since the release of Solaris 10 in 2005 was codenamed 'Nevada', and was derived from what was the OpenSolaris codebase. 'Solaris Nevada' is the codename for the next-generation Solaris OS to eventually succeed Solaris 10 and this new code was then pulled successively into new OpenSolaris 'Nevada' snapshot builds.[31] OpenSolaris is now discontinued and OpenIndiana forked from it.[33][34] A final build (b134) of OpenSolaris was published by Oracle (2010-Nov-12) as an upgrade path to Solaris 11 Express.

OpenZFS

In OpenZFS, the pool version is permanently set to 5000, signifying that the pool indicates new features by setting or unsetting ZFS feature flags rather than by incrementing the pool version.[35] The number 5000 was chosen because it is expected to never conflict with version numbers given by Oracle. Legacy version numbers still exist for pool versions 1–28.[36][37][38] Future on-disk format changes are enabled / disabled independently via these feature flags.

List of ZFS platforms

References

  1. ↑ "Oracle Announces Availability of Oracle Solaris 11.3". October 26, 2015. https://www.oracle.com/corporate/pressrelease/solaris-11-3-102615.html. Retrieved October 28, 2015. 
  2. ↑ "1.1 What about the licensing issue?". http://zfsonlinux.org/faq.html#WhatAboutTheLicensingIssue. Retrieved November 18, 2010. 
  3. ↑ Salus, Peter (1994). A Quarter Century of Unix. Addison-Wesley. pp. 199–200. ISBN 0-201-54777-5. 
  4. ↑ "What are SunOS and Solaris?". Knowledge Base. Indiana University Technology Services. May 20, 2013. http://kb.iu.edu/data/agjq.html. Retrieved November 10, 2014. 
  5. ↑ Brown, David. "A Conversation with Jeff Bonwick and Bill Moore". Association for Computing Machinery. http://queue.acm.org/detail.cfm?id=1317400. Retrieved 17 November 2015. 
  6. ↑ "ZFS: the last word in file systems". Sun Microsystems. September 14, 2004. Archived from the original on April 28, 2006. https://web.archive.org/web/20060428092023/http://www.sun.com/2004-0914/feature/. Retrieved April 30, 2006. 
  7. ↑ Matthew Ahrens (November 1, 2011). "ZFS 10 year anniversary". http://blog.delphix.com/matt/2011/11/01/zfs-10-year-anniversary/. Retrieved July 24, 2012. 
  8. ↑ Bonwick, Jeff (October 31, 2005). "ZFS: The Last Word in Filesystems". https://blogs.oracle.com/bonwick/en_US/entry/zfs_the_last_word_in. Retrieved June 22, 2013. 
  9. ↑ "Sun Celebrates Successful One-Year Anniversary of OpenSolaris". Sun Microsystems. June 20, 2006. http://www.sun.com/smi/Press/sunflash/2006-06/sunflash.20060620.1.xml. 
  10. ↑ Michael Singer (January 25, 2005). "Sun Cracks Open Solaris". InternetNews.com. http://www.internetnews.com/dev-news/article.php/3463621. Retrieved April 12, 2010. 
  11. ↑ "ZFS FAQ at OpenSolaris.org". Sun Microsystems. Archived from the original on May 15, 2011. https://web.archive.org/web/20110515061128/http://hub.opensolaris.org/bin/view/Community+Group+zfs/faq/. Retrieved May 18, 2011. "The largest SI prefix we liked was 'zetta' ('yotta' was out of the question)" 
  12. ↑ Jeff Bonwick (May 3, 2006). "You say zeta, I say zetta". Jeff Bonwick's Blog. Archived from the original on February 23, 2017. https://web.archive.org/web/20170223222515/https://blogs.oracle.com/bonwick/en_US/entry/you_say_zeta_i_say. Retrieved April 21, 2017. "So we finally decided to unpimp the name back to ZFS, which doesn't stand for anything." 
  13. ↑ "Oracle and NetApp dismiss ZFS lawsuits". theregister.co.uk. 2010-09-09. https://www.theregister.co.uk/2010/09/09/oracle_netapp_zfs_dismiss/. Retrieved 2013-12-24. 
  14. ↑ "The Linux Kernel, CDDL and Related Issues". February 26, 2016. https://www.softwarefreedom.org/resources/2016/linux-kernel-cddl.html. Retrieved March 30, 2016. 
  15. ↑ "GPL Violations Related to Combining ZFS and Linux". February 25, 2016. https://sfconservancy.org/blog/2016/feb/25/zfs-and-linux/. Retrieved March 30, 2016. 
  16. ↑ Petros Koutoupis (June 1, 2016). "ZFS: Finding Its Way to a Linux Near You?". Linux Journal. http://www.linuxjournal.com/content/zfs-finding-its-way-linux-near-you. Retrieved July 4, 2016. 
  17. ↑ Ryan Paul (June 9, 2010). "Uptake of native Linux ZFS port hampered by license conflict". https://arstechnica.com/information-technology/2010/06/uptake-of-native-linux-zfs-port-hampered-by-license-conflict/. Retrieved July 1, 2014. 
  18. ↑ "ZFS on Linux: Frequently asked questions: Licensing". May 26, 2016. https://github.com/zfsonlinux/zfs/wiki/FAQ#licensing. Retrieved July 3, 2016. 
  19. ↑ Simon Sharwood (April 21, 2016). "Ubuntu 16.04 LTS arrives today complete with forbidden ZFS". https://www.theregister.co.uk/2016/04/21/ubuntu_16_04_lts_launched/. Retrieved July 3, 2016. 
  20. ↑ Michael Larabel (October 6, 2015). "Ubuntu is Planning to Make The ZFS Filesystem a "Standard" Offering". Phoronix. https://www.phoronix.com/scan.php?page=news_item&px=Ubuntu-ZFS-Standard-Plans. Retrieved July 3, 2016. 
  21. ↑ James Bottomley (February 23, 2016). "Are GPLv2 and CDDL incompatible?". http://blog.hansenpartnership.com/are-gplv2-and-cddl-incompatible/. Retrieved July 3, 2016. 
  22. ↑ Steven Stallion / Oracle (August 13, 2010). "Update on SXCE". Iconoclastic Tendencies. http://sstallion.blogspot.com/2010/08/opensolaris-is-dead.html. 
  23. ↑ Alasdair Lumsden. "OpenSolaris cancelled, to be replaced with Solaris 11 Express". osol-discuss (Mailing list). Archived from the original on August 16, 2010. Retrieved November 24, 2014.
  24. ↑ Solaris still sorta open, but OpenSolaris distro is dead on Ars Technica by Ryan Paul (Aug 16, 2010)
  25. ↑ Garrett D'Amore (3 August 2010). "Illumos - Hope and Light Springs Anew - Presented by Garrett D'Amore". illumos.org. http://www.illumos.org/attachments/download/3/illumos.pdf. Retrieved 3 August 2010. 
  26. ↑ "Whither OpenSolaris? Illumos Takes Up the Mantle". Archived from the original on 26 September 2015. https://web.archive.org/web/20150926053916/http://www.linuxinsider.com/story/76669.html. 
  27. ↑ Garrett D'Amore (13 August 2010). "The Hand May Be Forced". http://gdamore.blogspot.com/2010/08/hand-may-be-forced.html. Retrieved 14 November 2013. 
  28. ↑ "illumos FAQs". illumos. http://wiki.illumos.org/display/illumos/illumos+FAQs. Retrieved 2013-09-24. 
  29. ↑ "LinuxCon 2013: OpenZFS" (PDF). 2013-09-17. http://events.linuxfoundation.org/sites/events/files/slides/OpenZFS%20-%20LinuxCon_0.pdf. Retrieved 2013-11-13. 
  30. ↑ "OpenZFS History". OpenZFS. http://open-zfs.org/wiki/History. Retrieved 2013-09-24. 
  31. ↑ 31.0 31.1 31.2 "While under Sun Microsystems' control, there were bi-weekly snapshots of Solaris Nevada (the codename for the next-generation Solaris OS to eventually succeed Solaris 10) and this new code was then pulled into new OpenSolaris preview snapshots available at Genunix.org. The stable releases of OpenSolaris are based off of these Nevada builds." Larabel, Michael. "It Looks Like Oracle Will Stand Behind OpenSolaris". Phoronix Media. https://www.phoronix.com/scan.php?page=news_item&px=ODQyOQ. Retrieved 21 November 2012. 
  32. ↑ "Oracle ZFS Storage Simulator download". Oracle Corporation. 2017. http://www.oracle.com/technetwork/server-storage/sun-unified-storage/downloads/sun-simulator-1368816.html. Retrieved January 12, 2018. 
  33. ↑ Ljubuncic, Igor (23 May 2011). "OpenIndiana — there's still hope". DistroWatch. http://distrowatch.com/weekly.php?issue=20110523#feature. 
  34. ↑ "Welcome to Project OpenIndiana!". Project OpenIndiana. 10 September 2010. http://openindiana.org/. Retrieved 14 September 2010. 
  35. ↑ "Features – OpenZFS – Feature flags". OpenZFS. http://open-zfs.org/wiki/Features#Feature_Flags. Retrieved 22 September 2013. 
  36. ↑ Siden, Christopher (January 2012). "ZFS Feature Flags". Delphix. p. 4. http://blog.delphix.com/csiden/files/2012/01/ZFS_Feature_Flags.pdf. Retrieved 2013-09-22. 
  37. ↑ "/usr/src/uts/common/sys/fs/zfs.h (line 338)". illumos (GitHub). https://github.com/illumos/illumos-gate/blob/master/usr/src/uts/common/sys/fs/zfs.h#L338. Retrieved 2013-11-16. 
  38. ↑ "/usr/src/uts/common/fs/zfs/zfeature.c (line 89)". illumos (GitHub). https://github.com/illumos/illumos-gate/blob/master/usr/src/uts/common/fs/zfs/zfeature.c#L89. Retrieved 2013-11-16. 

External links

  • Fork Yeah! The Rise and Development of illumos - slide show covering much of the history of Solaris, the decision to open source by Sun, the creation of ZFS, and the events causing it to be close sourced and forked after Oracle's acquisition.




Retrieved from "https://handwiki.org/wiki/index.php?title=Software:History_and_implementations_of_ZFS&oldid=3043853"