Record-oriented filesystem

Short description: File system where data is stored as collections of records

"Record-oriented file" redirects here. For this feature in the Distributed Data Management Architecture, see Record-oriented file (DDM).

In computer science, a record-oriented filesystem is a file system where data is stored as collections of records. This is in contrast to a byte-oriented filesystem, where the data is treated as an unformatted stream of bytes. There are several different possible record formats; the details vary depending on the particular system. In general the formats can be fixed-length or variable length, with different physical organizations or padding mechanisms; metadata may be associated with the file records to define the record length, or the data may be part of the record. Different access methods for records may be provided, for example records may be retrieved in sequential order, by key, or by record number.

Origin and characteristics

Record-oriented filesystems are frequently associated with mainframe operating systems, such as OS/360 and successors and DOS/360 and successors, and midrange operating systems, such as RSX-11 and VMS. However, they originated earlier in software such as Input/Output Control System (IOCS). Records, sometimes called logical records, are often written together in blocks, sometimes called physical records; this is the norm for direct access and tape devices, but files on unit record devices are normally unblocked, i.e., there is only one record per block.

Record-oriented filesystems can be supported on media other than direct access devices. A deck of punched cards can be considered a record-oriented file. A magnetic tape is an example of a medium that can support records of uniform length or variable length.

In a record file system, a programmer designs the records that may be used in a file. All application programs accessing the file, whether adding, reading, or updating records share an understanding of the design of the records. In DOS/360, OS/360, and their successors there is no restriction on the bit patterns composing the data record, i.e. there is no delimiter character; this is not always true in other software, e.g., certain record types for RCA File Control Processor (FCP) on the 301, 501, 601 and 3301.

The file comes into existence when a file create request is issued to the filesystem. Some information about the file may be included with the create request. This information may specify that the file has fixed-length records (all records are the same size) along with the size of the records. Alternatively, the specification may state that the records are of variable length, along with the maximum record length. Additional information including blocking factor, binary vs. text and the maximum number of records may be specified.

It may be permitted to read only the beginning of a record; the next sequential read returns the next collection of data (record) that the writer intended to be grouped together. It may also be permitted to write only the beginning of a record. In these cases, the record is padded with binary zeros or with spaces, depending on whether the file is recognized as a binary file or a text file.

Some operating systems require that library routines specific to the record format be included in the program. This means that a program originally expected to read a variable length record file cannot read a fixed length file. These operating systems must provide file system utilities for converting files between one format and another. This means copying the file (which requires additional storage space, time and coordination) may be necessary.

Other operating systems include various routines and associate the appropriate routine, based on the file organization, at execution time.

In either case, significant amounts of code to manage records must be provided in protected routines to ensure file integrity.

An alternative to a Record-oriented file is a stream file, in which the file system treats a file as an unstructured sequence of bytes. The applications may, but need not, impose a record structure. This approach significantly reduces the size and complexity of the library and reduces the number of utilities required to maintain files.

A common application convention for text files represented as streams is to use a new line delimiter to separate or terminate records, commonly CR, CRLF or LF. Unfortunately, the CPU time required to parse for the record delimiter is significant and the exclusion of the record delimiter pattern from the data is frequently undesirable.

An alternate convention is to include a length field in each record. The writer application is responsible for imposing any record structure and the reader application is responsible for separating out the records.

Advantages and costs

A record oriented file has several advantages. After a program writes a collection of data as a record the program that reads that record has the understanding of those data as a collection. Often a file will contain several related records in sequence; after the program reads the beginning of the sequence, the next sequential read returns the next collection of data (record) that the writer intended to be grouped together. Another advantage is that the record has a length and there is usually no restriction on the bit patterns composing the data record, i.e. there is no delimiter character.

There is usually a cost associated with record oriented files. For fixed length records, some records may have unused space, while for variable length records the delimiter or length field takes up space. Variable length blocks may have overhead due to delimiters or length fields. In addition, there is overhead imposed by the device. On a magnetic tape overhead typically takes the form of an inter-record gap. On a direct access device with fixed length sectors, there may be unused space in the last sector of a block. On a direct access device with variable length physical records, that overhead typically takes the form of metadata and inter-record gaps.

On a file composed of varying length records a maximum record length is defined to determine the size of the length metadata associated with each record.

A major advantage of record-oriented file systems is that they abstract files kept on paper in earlier times. A record might contain data associated with a particular, e.g., building, contact, employee, part, venue.

A second motivator for the idea of record orientation is that it is in some sense the more natural orientation for persistent storage on a non-volatile but slow physical storage device. Most physical storage devices can communicate only in units of a block. Significant portions of modern operating system kernels and associated device drivers are devoted to hiding the naturally structured and delimited (and in some sense a block is just a physical record) nature of physical storage devices. It is not coincidental that record oriented file systems arose earlier in the history of computing than byte-stream oriented file systems, when the capabilities for abstraction were far less.

See also

• Stream (computing)
• Data set (IBM mainframe)
• Files-11
• CMS file system
• ISAM/VSAM
• Distributed Data Management Architecture (DDM)
• Record-oriented file (DDM)