Software:Linear Tape File System

From HandWiki

The Linear Tape File System (LTFS) is a file system that allows files stored on magnetic tape to be accessed in a similar fashion to those on disk or removable flash drives. It requires both a specific format of data on the tape media and software to provide a file system interface to the data.

The technology, based around a self-describing tape format developed by IBM, was adopted by the LTO Consortium in 2010.

History

Magnetic tape data storage has been used for over 50 years, but typically did not hold file metadata in a form easy to access or modify independent of the file content data. Often external databases were used to maintain file metadata (file names, timestamps, directory hierarchy) to hold this data but these external databases were generally not designed for interoperability and tapes might or might not contain an index of their content. In Unix-like systems, there is the tar interoperable standard, but this is not well-suited to allow modification of file metadata independent of modifying file content data - and does not maintain a central index of files nor provide a filesystem interface or characteristics.

LTFS technology was first implemented by IBM as a prototype running on Linux and Mac OS X during 2008/2009. This prototype was demonstrated at NAB 2009. Based on feedback from this initial demonstration and experience within IBM the filesystem was overhauled in preparation for release as a product. The LTFS development team worked with the vendors of LTO tape products (HP and Quantum) to build support and understanding of the LTFS format and filesystem implementation leading up to the public release.

The LTFS Format Specification and filesystem implementation were released on April 12, 2010 with the support of IBM, HP, Quantum, and the LTO Consortium.[1]

LTFS v2.0.0 was released in March 2011, improving the text to clarify and remove ambiguity. It also added support for sparse files; persistent file identifiers; virtual extended attributes for filesystem metadata and control - and defined minimum and recommended blocksize values for LTFS volumes, for compatibility across various HBA hardware implementations.

Format specification

The ISO/IEC 20919:2016 standard defines the LTFS Format requirements for interchanged media that claims LTFS compliance. It defines the data format, independent of the physical storage media and the software commands format, to make data truly interchangeable. The ISO standard was prepared by SNIA. It is based on LTFS v2.2, and was adopted to ISO by a joint technical committee ISO/IEC JTC 1 Information Technology.

The SNIA workgroup continues to develop LTFS and release updates. Version 2.0.0 defines rules for how the version number may change in future, and how compatibility is maintained across varying implementations. All implementations must:

  • correctly read media that was compliant with any prior version
  • write media that is compliant with the version they claim compliance with
LTFS Format Specification History
Version Published Conforming Software
1.0 April 2010 IBM Long Term File System (LTFS) v1.0.0, v1.0.1
HP Linear Tape File System (LTFS) v1.0.0, v1.1.0
2.0.0 March 2011 IBM Linear Tape File System - Single Drive Edition (LTFS-SDE) v1.2.0,
Oracle StorageTek Linear Tape File System, Open Edition v1.0.0
2.1.0 October 2012 IBM: Linear Tape File System (LTFS) Format Specification
2.2.0 December 2013 Linear Tape File System (LTFS) Format Specification v2.2.0 SNIA Technical Position
2.3.0 March 2014 Linear Tape File System (LTFS) Format Specification v2.3.0 Revision 4
2.4.0 December 2017 Linear Tape File System (LTFS) Format Specification v2.4.0
2.5.0 May 2019 Linear Tape File System (LTFS) Format Specification v2.5.0

SNIA Technical Work Group

In August 2012, SNIA announced[2] that it was forming a TWG (Technical Work Group) to continue technical development of the specification. LTFS Format Specification v 2.1 is the baseline for the technical work and standards accreditation process; SNIA LTFS TWG members include HP, IBM, Oracle and Quantum.

Nature

While LTFS can make a tape appear to behave like a disk, it does not change the fundamentally sequential nature of tape. Files are always appended to the end of the tape. If a file is modified and overwritten or removed from the volume, the associated tape blocks used are not freed up, they are simply marked as unavailable and the used volume capacity is not recovered. Data is only deleted and capacity recovered if the whole tape is reformatted.[citation needed]

In spite of these disadvantages, there are several uses cases where LTFS formatted tape is superior to disk and other data storage technologies. While LTO seek times can range from 10 to 100 seconds, the streaming data transfer rate can match or exceed disk data transfer rates. Additionally, LTO cartridges are easily transportable and hold far more data than any other removable data storage format. The ability to copy a large file or a large selection of files (up to 1.5TB for LTO-5 or 2.5TB for LTO_6) to an LTFS formatted tape, allows easy exchange of data to a collaborator, or the saving of an archival copy.

Since LTFS is an open standard, LTFS formatted tapes are usable by a wide variety of computing systems.

Implementations

IBM Linear Tape File System - Single Drive Edition

IBM Linear Tape File System - Single Drive Edition
Developer(s)IBM
Initial releaseApril 2010
Stable release
2.2.2.0 (9802) / 28 October 2016; 7 years ago (2016-10-28)
Written inC
Operating systemLinux, macOS, Microsoft Windows
Platformx64, Intel x86 - 32-bit
Available inEnglish
TypeStorage software
LicenseGNU Lesser General Public License
Websiteibm.com/systems/storage/tape/ltfs/

The IBM Linear Tape File System - Single Drive Edition, (initially released as "IBM Long Term File System"), allows tapes to be formatted as an LTFS volume, and for these volumes to be mounted - and users and applications access files and directories stored on the tape directly, including drag-and-drop of files.

Operating systems supported by IBM LTFS-SDE

  • Red Hat Enterprise Linux (RHEL 5) with LTFS-SDE - 1.2.0 for Linux (support for RHEL 5 dropped in 2.2.2.0, last supported release 2.2.1.1)
  • Red Hat Enterprise Linux (RHEL 6) with LTFS-SDE
  • Red Hat Enterprise Linux (RHEL 7) with LTFS-SDE - 2.2.2.0 for Linux
  • SUSE Linux Enterprise Server (SLES) 11 SP 1 with LTFS-SDE 1.2.0 for Linux
  • Mac OS X 10.5.6 (Leopard), 10.6.7 (Snow Leopard) with LTFS-SDE - 1.2.0 for Mac OS X (Support dropped in version 2.2.2.0)
  • Mac OS X 10.7 (Lion), with LTFS-SDE - 1.2.5 for Mac OS X (Support dropped in version 2.2.2.0)
  • Mac OS X 10.8 (Mountain Lion), with LTFS-SDE - 1.3.0.2 for Mac OS X (Support dropped in version 2.2.2.0)
  • Mac OS X 10.9 (Mavericks), with LTFS-SDE - 2.2.0.0 for Mac OS X (Support dropped in version 2.2.2.0)
  • Mac OS X 10.10 (Yosemite), 10.11 (El Capitan) with LTFS-SDE - 2.2.2.0 for Mac OS X
  • Windows 7 with LTFS-SDE - Windows version 1.2.0
  • Windows 8 with LTFS-SDE - Windows version 1.3.0.1
  • Windows 8.1 with LTFS-SDE - Windows version 2.2.0.0
  • Windows Server 2008 R2 with LTFS-SDE - Windows version 1.2.2
  • Windows Server 2012 with LTFS-SDE - Windows version 1.3.0.1

IBM Linear Tape File System - Library Edition

IBM Linear Tape File System - Library Edition
Developer(s)IBM
Initial releaseJune 2011
Stable release
2.0.0 / 28 March 2011; 13 years ago (2011-03-28)
Written inC
Operating systemLinux
Platformx64, Intel x86 - 32-bit
Available inEnglish
TypeStorage software
LicenseProprietary commercial software
Websiteibm.com/systems/storage/tape/ltfs/

The IBM Linear Tape File System - Library Edition (LTFS-LE) product allows LTFS volumes to be used in a tape library. Each LTFS-formatted tape cartridge in the library appears as a separate folder under the filesystem mount point and the user or application can navigate into each of these folders to access the files stored on each tape. The LTFS-LE software automatically controls the tape library robotics to load and unload the necessary LTFS volumes.

Operating systems supported by IBM LTFS-LE

Oracle's StorageTek Linear Tape File System, Open Edition

Oracle's StorageTek Linear Tape File System, Open Edition
Developer(s)ORACLE
Initial release2011
Stable release
1.2.6
Written inC
Operating systemLinux
Platformx64, Intel x86 - 32-bit
Available inEnglish
TypeStorage software
LicenseLGPL-2.1, BSD
Websitewww.oracle.com/us/products/servers-storage/storage/tape-storage/ltfs/ and oss.oracle.com/projects/ltfs/

Oracle's free open source StorageTek Linear Tape File System (LTFS), Open Edition software[3] is claimed to be the first to store 8.5TB (native capacity) on a single cartridge. It supports Oracle’s midrange StorageTek LTO 5 and LTO 6 tape drives from HP and IBM as well as Oracle’s StorageTek T10000C and T10000D tape drives.[4] [5]

Oracle's StorageTek Linear Tape File System, Library Edition

Oracle’s StorageTek LTFS-LE software offering supports the StorageTek SL8500 Modular Library System, the StorageTek SL3000 Modular Library System, and the StorageTek SL150 Modular Tape.[6]

HP Linear Tape File System

HP Linear Tape File System
Developer(s)HP
Stable release
1.1.0 / 29 November 2010; 13 years ago (2010-11-29)
Written inC
Operating systemLinux, Mac OS X
Platformx64, Intel x86 - 32-bit
LicenseGNU Lesser General Public License
Websitehpe.com/storage/storeopen

The HP Linear Tape File System (HP LTFS) is HP's implementation. It is a free open source software application.

Operating systems supported by HP LTFS

Quantum Linear Tape File System

Quantum LTFS
Developer(s)Quantum Corporation
Operating systemWindows, Linux, Mac OS X
LicenseGNU Lesser General Public License
Websitewww.quantum.com/products/tapedrives/ltoultrium/lto-5/ltfs/index.aspx

Quantum Corporation provides an LTFS product with Windows, Linux and Mac OS X support.

The Scalar LTFS Appliance is a file system that presents a Quantum tape library as an NAS share. This appliance makes files viewable as if they resided on a local disk and allows users to drag and drop files directly to and from a tape cartridge.

LTFS compatible products

Enterprise Drives

  • IBM TS1140 Tape Drive
IBM 4 TB Native Capacity Tape Drive [7]
  • Oracle StorageTek T10000C Tape Drive
Oracle 5 TB Native Capacity Tape Drive [8]

LTO drives

  • HP StorageWorks LTO-5 Ultrium 3000
HP LTO5 external half height tape drive, 6 Gbit/s SAS
  • HP StorageWorks LTO-5 Ultrium 3280
HP LTO5 full height tape drive, 6 Gbit/s SAS
  • IBM System Storage TS2250 Tape Drive
IBM LTO5 external stand-alone half height tape drive, 6 Gbit/s SAS interface[9]
  • IBM System Storage TS2350 Tape Drive
IBM LTO5 external stand-alone full height tape drive, 6 Gbit/s SAS interface[10]
  • IBM LTO5 FH FC Tape Drive
IBM LTO5 internal full height tape drive, 8 Gbit/s Fiber Channel interface[11]
  • IBM LTO5 FH SAS Tape Drive
IBM LTO5 internal full height tape drive, 6 Gbit/s SAS interface[11]
  • IBM LTO5 HH FC Tape Drive
IBM LTO5 internal half height tape drive, 8 Gbit/s Fiber Channel interface[11]
  • IBM LTO5 HH SAS Tape Drive
IBM LTO5 internal half height tape drive, 6 Gbit/s SAS interface[11]
  • Quantum LTO5 HH SAS Tape Drive
Quantum LTO5 external half height tape drive, 6 Gbit/s SAS interface[12]
  • Quantum LTO5 FH SAS Tape Drive
Quantum LTO5 internal half height tape drive, 6 Gbit/s SAS interface[12]

Appliances

  • Storage Heaven TapeMaster LTFS | LTO Ultrium Tape Cloning Appliances
The TapeMaster [13] LTFS | LTO Ultrium Tape migration and duplication system features high speed tape cloning without the need for a computer or application software. TapeMaster LTO cloning system comes in manual or fully automated configurations, includes support for barcodes, encrypted tapes and supports all generation of LTO Ultrium tapes.
  • 1 Beyond Wrangler LTO-5 Offload Stations
The 1 Beyond[14] Wrangler LTO-5 Offload Stations is an appliance to store broadcast content directly on tape via LTFS.
  • Cache-A’s LTO-5 based products
Cache-A Corporation[15] demonstrated one of the early integrations of the LTFS format on LTO Gen5 using IBM's LTFS codebase[16] at NAB 2010. Cache-A is continuing this development with the HP codebase[17] to provide LTFS-based archive appliances that provide access to this technology without requiring any client-side software.
  • StrongBox LTFS tape-based archive
StrongBox Data[18] formerly known as Crossroads[19] announced on April 11th, 2011,[20] to launch in a new tape-based archive product that leverages the Linear Tape File System (LTFS) technology. This solution[buzzword] will serve as an online archive for tier3(Tertiary storage) data, presenting itself as disk storage, but incorporating both disk for fast file storage and retrieval, and physical LTO tape for cost-effective, long-term capacity storage. Products available are StrongBOX T1 and StrongBOX T3.
  • FOR-A LTR-110HS/120HS Video Archive Recorder
The FOR-A[21] LTR-100HS digital video recorder device is using IBM LTFS formatted tape cartridges. It was introduced at the NAB 2010 and won the IBC 2010 Pick Hit Awards from Broadcast Engineering. Its successor is the FOR-A LTR-120HS [AVC-Intra/DVCPRO Model], which supports additional MXF format
  • Nodeum LTFS Appliance
Nodeum[22] is a Software Defined based solution. It is a LTFS Compliant solution.[23] The solution is Linux based, the installation can be done with an Ansible playbook into any x86 commodity hardware. The Appliance provides a virtualized file system accessible with NAS protocols ; the architecture is composed by disk - flash for caching and a Tape Library manager for extending the capacity on LTO tapes. The Appliance virtualized storage can also be presented as a S3 endpoint, allowing S3 access of data stored on tapes.
  • Spectra Logic BlackPearl
The BlackPearl[24] appliance is a deep storage gateway that allows reading and writing LTFS-formatted tapes through a S3 interface.

ISVs (Independent Software Vendors) supporting LTFS

  • BRU PE
TOLIS Group's[25] BRU Producer's Edition version 2.3.0 provides native LTFS (Linear Tape Filesystem Support) for OS X users.[26]
  • FlashNet
SGL[27] (Software Generation Limited) a UK-based broadcast archive software solutions[buzzword] company, was the first Independent Software Vendor (ISV), which announced[28] their support of LTFS with their FlashNet product.
  • Masstech Group Inc.
Masstech Group Inc.,[29] a Toronto, Canada-based developer of media asset management, archiving, workflow automation and transcoding solutions[buzzword] for the media and entertainment industry, announced LTFS support across the company's solution[buzzword] portfolio in 2012.[30]
  • Open Source Storage Inc.
Open Source Storage Inc.,[31] a Colorado Corporation, develops, installs and supports an open-source implementation of LTFS with extensions for support of tape libraries so that one or more entire tape libraries appear as a single file system name space, with automatic copying and retrieval of files from LTFS volumes. Two products, OSVault and Wellspring, are based on the IBM LTFS SE baseline, the media is interchangeable with any other LTFS implementation. First deployed in 2006, the LTFS filesystem support and LTFS autmounter support was added in 2012 and the first production system went live in January 2013, with over 100 production sites in Broadcast/Entertainment and Medical Applications in operation today.[citation needed]
  • PreRollPost
PreRollPost[32] can be used with any LTFS compliant tape drive within Macintosh OSX 10.9 and higher or Windows 10, 8, or 7. Backup, scheduling, and retrieving are all possible with PreRollPost which also uses checksum technology to ensure all archives match 100%. No formatting, software, nor other hardware is required. A single click is required to install everything for LTFS. Backup can be used with any files ie NLE project files or entire hard drives. PreRollPost also supports backing up camera originals with an index saved on the local hard drive or LTO tape. Compatible with Sony's Optical Disc Archive, mLogic's mTape, MagStor by MagNext and the LT60 from Unitex.
  • QStar Archive Manager with LTFS Volume Spanning
QStar[33] announced on 19 July 2012,[34] QStar Unveils Industry’s First Software-Based LTFS Volume-Spanning and File-Spanning Capability for Tape Libraries. With the use of QStar’s LTFS volume-spanning technology, all media within the tape library can be seen as one or more ever-growing network share(s). New LTFS media is automatically added to the set as the previous media becomes full. This approach eliminates the constant stopping and manual reconfiguration required by standard LTFS methodologies as a tape reaches capacity.
  • XenData
XenData’s entire digital video archive product range supports LTFS,[35] offering a number of advantages compared to other LTFS solutions.[buzzword][36]
  • YoYotta
YoYotta create production and post workflow software for archiving, backup and restore using LTFS on LTO-7, LTO-6 or LTO-5 tapes. YoYottaID LTFS controls two standalone LTO drives. YoYottaID Automation controls an LTO library and allows automated creation of multi-tape LTFS archives.[37]

LTFS projects

  • Thought Equity Motion[38] is executing a major film digitization and preservation project for the EYE Film Institute Netherlands. The project involves scanning more than 150 million discrete DPX files and storing them on LTO Gen5 using the LTFS format.[39] More than 1 petabyte of film will be scanned and archived over two years (2010–2012).

Industry recognition

  • IBM LTFS technology received a Pick Hit Award from Broadcast Engineering at NAB 2011.[40]
  • IBM and FOX Networks received an Engineering Emmy Award in 2011 for a project that uses LTFS to store, exchange, and archive video content.[41]
  • IBM received the 2011 Hollywood Post-Alliance (HPA) Engineering Excellence Award.[42]

References

  1. ↑ NAB-2010
  2. ↑ "SNIA Announces Linear Tape File System Technical Work Group"
  3. ↑ Oracle StorageTek Linear Tape File System, Open Edition Webpage
  4. ↑ Oracle StorageTek T10000C Tape Drive Press Release
  5. ↑ Oracle Introduces StorageTek T10000D Tape Drive
  6. ↑ Oracle StorageTek LTFS, Library Edition Press Release
  7. ↑ "IBM System Storage TS1140 Tape Drive Model E07 delivers higher performance, reliability, and capacity". IBM. http://www-01.ibm.com/common/ssi/ShowDoc.wss?docURL=/common/ssi/rep_ca/7/897/ENUS111-087/index.html&lang=en&request_locale=en. Retrieved 4 October 2012. 
  8. ↑ Oracle StorageTek T10000C Tape Drive Webpage
  9. ↑ IBM TS2250 Webpage
  10. ↑ IBM TS2350 Webpage
  11. ↑ 11.0 11.1 11.2 11.3 IBM Ultrium tape drives, libraries, and autoloaders
  12. ↑ 12.0 12.1 Quantum LTO-5 Webpage
  13. ↑ "TapeMaster LTO Data Tape Duplicator and Migration System" (in en). http://www.storageheaven.com/TapeMaster-LTO-Data-Tape-Copier-p/1099.htm. 
  14. ↑ 1 Beyond - Homepage
  15. ↑ Cache-A - Homepage
  16. ↑ IBM LFTS Linux Codebase
  17. ↑ "HP LTFS Linux Codebase". http://h20000.www2.hp.com/bizsupport/TechSupport/SoftwareIndex.jsp?lang=en&cc=us&prodNameId=4249245&prodTypeId=12169&prodSeriesId=4249221&swLang=8&taskId=135&swEnvOID=4006. 
  18. ↑ StrongBox Data - Homepage
  19. ↑ Crossroads - Homepage
  20. ↑ Crossroads - Press Release, Austin, TX, April 11, 2011 : Plans to Launch Archive Product that Ushers in a New Era for Tape
  21. ↑ FOR-A - Homepage
  22. ↑ "Nodeum LTFS Appliance". https://www.nodeum.io/ltfs-appliance. 
  23. ↑ "LTFS COMPLIANCE VERIFICATION". https://www.lto.org/technology/ltfs/ltfs-compliance-verification/. 
  24. ↑ Spectra Logic BlackPearl
  25. ↑ Tolisgroup - Homepage
  26. ↑ TOLIS Group - Press Release, Las Vegas, NV — April 11, 2011: BRU PRODUCER'S EDITION (BRU PE) SUPPORTS LTFS
  27. ↑ SGL - Homepage
  28. ↑ SGL - Press Release, Fareham, UK - February 9, 2011: SGL announces LTFS implementation transforming digital media workflows
  29. ↑ Masstech Group Inc.-Homepage
  30. ↑ Masstech provides archive support for IBM’s groundbreaking Linear Tape File System Press Release
  31. ↑ "Back To The Future - Open Source Storage (Re)launches", JAN 27, 2014, Ben Kepes, Forbes
  32. ↑ Imagine Products Inc.-Homepage
  33. ↑ QStar - Homepage
  34. ↑ QStar - Press Release, Navarre, Florida:
  35. ↑ XenData - Products
  36. ↑ XenData First to Launch LTO Video Archives with Multiple Format File System Support Including LTFS and tar
  37. ↑ YoYotta - Homepage
  38. ↑ Thought Equity Motion - Homepage
  39. ↑ Thought Equity Motion - Press Release: Film Archive Digitization and Preservation Project for EYE Film Institute Netherlands
  40. ↑ NAB 2011 Pick Hit Announcement
  41. ↑ "IBM Almaden Blog". http://ibmresearchalmaden.blogspot.com/2011/10/we-won-emmy.html. 
  42. ↑ HPA Award Announcement

External links

Implementations:




Retrieved from "https://handwiki.org/wiki/index.php?title=Software:Linear_Tape_File_System&oldid=322245"