Engineering:Tagged Command Queuing

Tagged Command Queuing (TCQ) is a technology built into certain ATA and SCSI hard drives. It allows the operating system to send multiple read and write requests to a hard drive. ATA TCQ is not identical in function to the more efficient Native Command Queuing (NCQ) used by SATA drives. SCSI TCQ does not suffer from the same limitations as ATA TCQ. Without TCQ, an operating system was limited to sending one request at a time. To boost performance, the OS had to determine the order of the requests based on its own – possibly incorrect – perspective of the hard drive activity (otherwise known as I/O scheduling). With TCQ, the drive can make its own decisions about how to order the requests (and in turn relieve the operating system from having to do so). Thus TCQ can improve the overall performance of a hard drive if it is implemented correctly.

Overview

For increased efficiency the sectors should be serviced in order of proximity to the current head position, not the order received. The queue is constantly receiving new requests, fulfilling and removing existing requests, and re-ordering the queue according to the current pending read/write requests and the changing position of the head. The exact reordering algorithm may depend upon the controller and the drive itself, but the host computer simply makes requests as needed, leaving the controller to handle the details.

This queuing mechanism is sometimes referred to as "elevator seeking", as the image of a modern elevator in a building servicing multiple calls and processing them to minimise travel illustrates the idea well.

If the buttons for floors 5, 2, and 4 are pressed in that order with the elevator starting on floor 1, an old elevator would go to the floors in the order requested. A modern elevator processes the requests to stop at floors in the logical order 2, 4, and 5, without unnecessary travel. Non-queueing disk drives service the requests in the order received, like an old elevator; queueing drives service requests in the most efficient order. This may improve performance slightly in a system used by a single user, but may dramatically increase performance in a system with many users making widely varied requests on the disk surface.

Comparison of SCSI TCQ, ATA TCQ, and SATA NCQ

SCSI TCQ

SCSI TCQ was the first popular version of TCQ and is still popular today. It allows tasks to be entered into a queue using one of three different modes:

• head of queue
• ordered
• simple

In head of queue mode, unique to SCSI TCQ, a task is pushed into the front of a queue, ahead of all other tasks including other pending head of queue tasks. This mode is not used much because it can cause resource starvation when abused.

In ordered mode, a task must execute after all older tasks have completed and before all newer tasks begin to execute (excluding newer head of queue tasks).

Simple mode allows tasks to execute in any order that does not violate the constraints on the tasks in the other two modes. After a command in a task is completed, a notification is sent by the device that completed the command to the host bus adapter.

Whether or not SCSI TCQ causes massive interrupt overhead depends on the bus being used to connect the SCSI host bus adapter. On Conventional PCI, PCI-X, PCI Express, and other buses that permit it, first party DMA allows for low interrupt overhead. The older ISA bus required a SCSI host adapter to generate an interrupt to cause the CPU to program the third-party DMA engine to perform a transfer, and then required another interrupt to notify the CPU that a task in the queue was finished, causing high CPU overhead.

SCSI TCQ Tag Length

The SCSI-3 protocol permits 64 bits to be used in the tag field, allowing up to 2⁶⁴ tasks in one task set to be issued before requiring that some of them complete before any more commands be issued. However, different protocols that implement the SCSI protocol might not permit the use of all 64 bits. For example, older parallel SCSI permits 8 bits of tag bits, iSCSI permits up to 32 tag bits, and Fibre Channel permits up to 16 bits of tag with tag 0xFFFF reserved. This flexibility allows the designer of a protocol to trade off queuing ability against cost. Networks that can be large, such as iSCSI networks, benefit from more tag bits to deal with the larger number of disks in the network and the larger latencies such large networks generate, while smaller-scale networks, such as parallel SCSI chains, do not have enough disks or latency to need many tag bits and can save money by using a system supporting fewer bits.

ATA TCQ

ATA TCQ was developed in attempt to bring the same benefits as SCSI to ATA drives. It is available in both Parallel and Serial ATA.

This effort was not very successful because the ATA bus started out as a reduced-pin-count ISA bus. The requirement for software compatibility made ATA host bus adapters act like ISA bus devices without first party DMA. When a drive was ready for a transfer, it had to interrupt the CPU, wait for the CPU to ask the disk what command was ready to execute, respond with the command that it was ready to execute, wait for the CPU to program the host bus adapter's third-party DMA engine based on the result of that command, wait for the third party DMA engine to execute the command, and then had to interrupt the CPU again to notify it when the DMA engine finished the task so that the CPU could notify the thread that requested the task that the requested task was finished. Since responding to interrupts uses CPU time, CPU utilization rose quickly when ATA TCQ was enabled. Also, since interrupt service time can be unpredictable, there are times when the disk is ready to transfer data but is unable to do so because it must wait for a CPU to respond to the interrupt so that the CPU knows that it needs to program the third party DMA engine.

Therefore, this standard was rarely implemented because it caused high CPU utilization without improving performance enough to make this worthwhile. This standard allows up to 32 outstanding commands per device.

SATA NCQ

SATA NCQ is a modern standard which drastically reduces the number of required CPU interrupts compared to ATA TCQ. Like ATA TCQ, it allows up to 32 outstanding commands per device, but was designed to take advantage of the ability of SATA host bus adapters that are not emulating parallel ATA behavior to support first party DMA. Instead of interrupting the CPU before the task to force it to program the host bus adapter's DMA engine, the hard drive tells the host bus adapter which command it wants to execute, causing the host bus adapter to program its integrated first-party DMA engine with the parameters that were included in the command that was selected by the hard drive when it was first issued, and then the DMA engine moves the data needed to execute the command. To further reduce the interrupt overhead, the drive can withhold the interrupt with the task completed messages until it gathers many of them to send at once, allowing the operating system to notify many threads simultaneously that their tasks have been completed. If another task completes after such an interrupt is sent, the host bus adapter can concatenate the completion messages together if the first set of completion messages has not been sent to the CPU. This allows the hard disk firmware design to trade off disk performance against CPU utilization by determining when to withhold and when to send completion messages.

External links

• Can Command Queuing Turbo Charge SATA? by Patrick Schmid, and Achim Roos of Tom's Hardware Guide