Engineering:ISCSI Conformance Testing and Testing Tool Requirement

From HandWiki

iSCSI conformance testing is testing to determine whether an iSCSI Initiator/Target meets the iSCSI standard.

SCSI and iSCSI Protocol

The Small Computer System Interface (SCSI) is a family of protocols for communicating with I/O devices, especially storage devices. SCSI is based on client-server model. SCSI clients, called "initiators", issue SCSI commands to request services from components, logical units of a server known as a "target". A "SCSI transport" maps the client-server SCSI protocol to a specific interconnect. An Initiator is one endpoint of a SCSI transport and a target is the other endpoint.

Internet Small Computer System Interface (iSCSI) protocol uses TCP/IP as SCSI transport. By transporting SCSI packets over TCP/IP, iSCSI provides an interoperable solution which can take advantage of existing Internet infrastructure, Internet management facilities, and address distance limitations. Unlike traditional Fibre Channel, which requires special-purpose cabling, iSCSI can run on the existing network infrastructure. It has become a popular storage area network (SAN) protocol because of its scalability and economic benefits. The protocol is defined in RFC 3720 and updated in RFC 5048.

iSCSI Conformance Testing

With the moving of SCSI transport layer from the traditional reliable bus structure to the far more unreliable TCP/IP network, a fairly reasonable amount of complexity has been introduced into iSCSI:

  • Login/Logout processes are needed.
  • Security/Operational parameters need to be negotiated during login.
  • CHAP or other authentication method may be used during login.
  • Multiple sessions/connections may be required.
  • Multiple error recovery level may be implemented.
  • ...

In one word, although the economic cost of iSCSI may be lower than others, the protocol itself is not that simple.

Since the approval of the protocol, iSCSI has been steadily gaining acceptance among end users and storage vendors. Nowadays, there exist a large number of iSCSI initiator/target implementations on the market, either commercial or free/open source. These implementations vary in functionalities and protocol conformance.

Conformance testing is testing to determine whether a system meets some specified standard. It is often performed by external organizations, sometimes the standards body itself, to give greater guarantees of compliance. Products tested in such a manner are then advertised as being certified by that external organization as complying with the standard. For iSCSI protocol (or other data networking and storage protocols), The University of New Hampshire InterOperability Laboratory (UNH-IOL) is one of the premier neutral, third-party laboratory who offers test suites for iSCSI conformance testing.

As the introduction part in these test suites says:

These tests are designed to determine if an iSCSI product conforms to specifications defined in both IETF RFC 3720 iSCSI (hereafter referred to as the “iSCSI Standard”) as well as updates as contained in IETF RFC 5048 iSCSI Corrections and Clarifications RFC (hereafter referred to as “iSCSI Corrections and Clarifications”). Successful completion of all tests contained in this suite does not guarantee that the tested device will successfully operate with other iSCSI products. However, when combined with satisfactory operation in the IOL’s interoperability test bed, these tests provide a reasonable level of confidence that the Device Under Test (DUT) will function properly in many iSCSI environments.

iSCSI conformance testing mainly contains the following aspects:

  • Login Phase Conformance
Test negotiation key-value pairs; test various flags: CSG, NSG, C bit, T bit...
  • Full Feature Phase Conformance
Test sequence numbers (CmdSN, DataSN, R2TSN); test SCSI Command/Response, Task Management Function Request/Response, Text Request/Response, NOP-In/NOP-Out, SNACK Request...
  • CHAP Conformance
Test CHAP fields: CHAP_A, CHAP_C, CHAP_I, CHAP_N, CHAP_R...
  • Multi Connection Conformance
Test multi-connection support in connection creation, termination, parameter negotiation, sequence number and task management...
  • Error Recovery Conformance
Test Initiator/Target's error recovery support

iSCSI Conformance Testing Common Scenarios

Basically two types of actions could be taken to test whether the DUT meets the iSCSI protocol:

  • We do correct things and see if the DUT behaves correctly.
    In this case, the main task is to send a correct sequence of iSCSI PDUs to the DUT, and to verify the DUT can respond with correct PDUs/PDU fields.
  • We do wrong things and see if the DUT can detect and behave accordingly.
    In this case, we must modify the PDU sequence sent to the DUT to some extent (e.g., change CmdSN of a command, set an invalid data digest...), and verify the DUT can react according to the protocol (e.g., send a Reject PDU, close the connection...).

Some common scenarios can be found in iSCSI conformance testing:

  • iSCSI functional test
To test whether an iSCSI task can be executed correctly (Task Management Function, NOP-In/NOP-Out Ping, multiple connections management...)
  • PDU format error
For example, this is an error if the W and F bit are both set to 0 in a SCSI Command PDU.
  • PDU session/stage error
For example, this is an error if a SCSI Command PDU appears in login stage or discovery session.
  • Header/Data digest error
Digest error can cause the Target/Initiator to send Reject/SNACK PDU.
  • Sequence error
CmdSN/StatSN/DataSN/R2T each has different scope and different rules.
  • Negotiation error
Text parameters should be in key=value format, followed by one NULL ("0x00") delimiter. Each text parameter also has its own stage and negotiation rules. For example, FirstBurstLength must not exceed MaxBurstLength, SendTargets should appear only in Full Feature Phase, etc.
  • Data length error
Too much or too little data is sent to the Target.
  • Lost PDU
Drop an incoming PDU then SNACK for it, drop an outgoing PDU then retry it.

Most of the above scenarios can be emulated by modifying the content of the output PDU. For example, to emulate a data digest error, we just need to change the DataDigest field in the PDU.

iSCSI Conformance Testing Tool Requirement

An iSCSI conformance testing tool can be viewed as an iSCSI Initiator emulator (to test iSCSI Target) or an iSCSI Target emulator (to test iSCSI Initiator). A good iSCSI conformance testing tool should meet the following requirements:

  • Executable test scripts
An executable script is needed for each test case. It should be written according to test case description which defines the PDU sequence and verification points.
  • An iSCSI Engine core
The iSCSI Engine core is in charge of sending/receiving iSCSI PDUs to/from DUT (Device Under Test). On one side, it accepts input from test script, sending PDUs or verifying check points; on the other, it should understand iSCSI protocol to a certain level so that necessary automation can be achieved, e.g., the automatic generation of iSCSI PDUs (Login/Text Request during negotiation, Nop-Out ping response, ) and some PDU fields (ITT, TTT, CmdSN, data, digest...).
  • PDU format validation
For received PDUs, iSCSI Engine core should do validation check to detect malformed PDU.
  • Ability to modify PDU's content
Malformed PDUs are often sent to DUT to simulate PDU format error or digest error.
  • Ability to drop PDU
PDU could be dropped to simulate lost PDU during transmission.
  • Ability to maintain multiple sessions/connections
Just like Error Recovery, multiple sessions/connections support is an advanced field in iSCSI protocol.
  • Result and log
Pass or Fail of a test case should have obvious indication. Detailed log is needed for analysis. It would be better to have a graphical interface to analyze PDU dump files.
  • A Batch system
A batch system is needed to manage test suites and test cases so that large number of test scripts can run automatically without user intervention.

Available iSCSI Conformance Testing Tools

  • UNH-IOL test service
    The iSCSI consortium provide a service to test iSCSI products and software from both interoperability and conformance perspectives. Membership is needed for cutting edge testing equipment and IOL testing tools.
  • iSCSISim
    Open-source software. iSCSISim is also an iSCSI initiator emulator to test iSCSI targets. Written in Python, its capabilities include SCSI command injection, error insertion, and recovery testing.
  • libiscsi
    Open-source software. libiscsi is a userspace library that implements an iSCSI initiator. Written in C it comes with an iSCSI/SCSI test-suite called iscsi-test-cu that can test a target's conformance.
  • Calsoft’s iSCSI Protocol Conformance Test Suite
    Commercial software. The test suite helps in iSCSI protocol conformance for any iSCSI target implementation.

See also



Retrieved from "https://handwiki.org/wiki/index.php?title=Engineering:ISCSI_Conformance_Testing_and_Testing_Tool_Requirement&oldid=147753"