Expanded memory

Short description: System of bank switching in DOS memory management

Several expanded-memory pages are bank-switched in the page frame, part of the upper memory area.

In DOS memory management, expanded memory is a system of bank switching that provided additional memory to DOS programs beyond the limit of conventional memory (640 KiB).

Expanded memory is an umbrella term for several incompatible technology variants. The most widely used variant was the Expanded Memory Specification (EMS), which was developed jointly by Lotus Software, Intel, and Microsoft, so that this specification was sometimes referred to as "LIM EMS". LIM EMS had several versions. The first widely implemented version was EMS 3.2, which supported up to 8 MiB of expanded memory and uses parts of the address space normally dedicated to communication with peripherals (upper memory) to map portions of the expanded memory. EEMS, an expanded-memory management standard competing with LIM EMS 3.x, was developed by AST Research, Quadram and Ashton-Tate ("AQA"); it could map any area of the lower 1 MiB. EEMS ultimately was incorporated in LIM EMS 4.0, which supported up to 32 MiB of expanded memory and provided some support for DOS multitasking as well. IBM, however, created its own expanded-memory standard called XMA.

The use of expanded memory became common with games and business programs such as Lotus 1-2-3 in the late 1980s through the mid-1990s, but its use declined as users switched from DOS to protected-mode operating systems such as Linux, IBM OS/2, and Microsoft Windows.

Background

A section of the lower 1 MiB address space provides a "window" into several megabytes of Expanded Memory

The 8088 processor of the IBM PC and IBM PC/XT could address one megabyte (MiB, or 2²⁰ bytes) of memory. It inherited this limit from the 20-bit external address bus (and overall memory addressing architecture) of the Intel 8086. The designers of the PC allocated the lower 640 KiB (655360 bytes) of address space for read-write program memory (RAM), called conventional memory, and the remaining 384 KiB of memory space was reserved for uses such as the system BIOS, video memory, and memory on expansion peripheral boards.

Even though the IBM PC AT, introduced in 1984, used the 80286 chip that could address up to 16 MiB of RAM as extended memory, it could only do so in protected mode. The scarcity of software compatible with protected mode (no standard DOS applications could run in it) meant that the market was still open for another solution.

To make more memory accessible, a bank switching scheme was devised, where only selected parts of the additional memory would be accessible at any given time. Originally, a single 64 KiB (2¹⁶ bytes) window of memory, called a page frame, was used; later this was made more flexible. Programs had to be written in a specific way to access expanded memory. The window between conventional memory and expanded memory could be adjusted to access different locations within the expanded memory.

A first attempt to use a bank switching technique was made by Tall Tree Systems with their JRAM boards, but these did not catch on. (Tall Tree Systems later made EMS-based boards using the same JRAM brand.)

Expanded Memory Specification (EMS)

Lotus Development, Intel, and Microsoft cooperated to develop the EMS standard (aka LIM EMS). The first publicly available version of EMS, version 3.0 allowed access of up to 4 MiB of expanded memory. This was increased to 8 MiB with version 3.2 of the specification. The final version of EMS, version 4.0 increased the maximum amount of expanded memory to 32 MiB and supported additional functionality.

Microsoft thought that bank switching was an inelegant and temporary, but necessary stopgap measure. Slamming his fist on the table during an interview Bill Gates said of expanded memory, "It's garbage! It's a kludge! … But we're going to do it". The companies planned to launch the standard at the Spring 1985 COMDEX, with many expansion-card and software companies announcing their support.

The first public version of the EMS standard, called EMS 3.0 was released in 1985; EMS 3.0, however, saw almost no hardware implementations before being superseded by EMS 3.2. EMS 3.2 used a 64 KiB region in the upper 384 KiB (upper memory area) divided into four 16 KiB pages, which could be used to map portions of the expanded memory.

In turn, EMS 3.2 was improved upon by a group of three other companies: AST Research, Quadram and Ashton-Tate, which created their own Enhanced EMS (EEMS) standard. EEMS allowed any 16 KiB region in lower RAM to be mapped to expanded memory, as long as it was not associated with interrupts or dedicated I/O memory such as network or video cards. Thus, entire programs could be switched in and out of the extra RAM. EEMS also added support for two sets of mapping registers. These features were used by early DOS multitasker software such as DESQview. Released in 1987, the LIM EMS 4.0 specification incorporated practically all features of EEMS.

A new feature added in LIM EMS 4.0 was that EMS boards could have multiple sets of page-mapping registers (up to 64 sets). This allowed a primitive form of DOS multitasking. The caveat was, however, that the standard did not specify how many register sets a board should have, so there was great variability between hardware implementations in this respect.

The Expanded Memory Specification (EMS) is the specification describing the use of expanded memory. EMS functions are accessible through software interrupt 67h. Programs using EMS must first establish the presence of an installed expanded memory manager (EMM) by checking for a device driver with the device name EMMXXXX0.

Expanded Memory Adapter (XMA)

IBM developed their own memory standard called Expanded Memory Adapter (XMA); the IBM DOS driver for it was XMAEM.SYS. Unlike EMS, the IBM expansion boards could be addressed both using an expanded memory model and as extended memory. The expanded memory hardware interface used by XMA boards is, however, incompatible with EMS, but a XMA2EMS.SYS driver provided EMS emulation for XMA boards. XMA boards were first introduced for the 1986 (revamped) models of the 3270 PC.

Implementations

Expansion boards

Emulex Persyst 4 MiB ISA memory board

This insertion of a memory window into the peripheral address space could originally be accomplished only through specific expansion boards, plugged into the ISA expansion bus of the computer. Famous 1980s expanded memory boards were AST RAMpage, IBM PS/2 80286 Memory Expansion Option, AT&T Expanded Memory Adapter and the Intel Above Board. Given the price of RAM during the period, up to several hundred dollars per MiB, and the quality and reputation of the above brand names, an expanded memory board was very expensive.

Motherboard chipsets

Later, some motherboard chipsets of Intel 80286-based computers implemented an expanded memory scheme that did not require add-on boards, notably the NEAT chipset. Typically, software switches determined how much memory should be used as expanded memory and how much should be used as extended memory.

Device drivers

An expanded-memory board, being a hardware peripheral, needed a software device driver, which exported its services. Such a device driver was called expanded-memory manager. Its name was variable; the previously mentioned boards used REMM.SYS (AST), PS2EMM.SYS (IBM), AEMM.SYS (AT&T) and EMM.SYS (Intel) respectively. Later, the expression became associated with software-only solutions requiring the Intel 80386 processor, for example Quarterdeck's QEMM, Qualitas' 386^MAX or the default EMM386 in MS-DOS, PC DOS and DR-DOS.

Software emulation

Beginning in 1986, the built-in memory management features of Intel 80386 processor freely modeled the address space when running legacy real-mode software, making hardware solutions unnecessary. Expanded memory could be simulated in software.

The first software expanded-memory management (emulation) program was CEMM, available in September 1986 as a utility for the Compaq Deskpro 386. A popular and well-featured commercial solution was Quarterdeck's QEMM. A contender was Qualitas' 386^MAX. Functionality was later incorporated into MS-DOS 4.01 in 1989 and into DR DOS 5.0 in 1990, as EMM386.

Software expanded-memory managers in general offered additional, but closely related functionality. Notably, they allowed using parts of the upper memory area (UMA) (the upper 384 KiB of real-mode address space) called upper memory blocks (UMBs) and provided tools for loading small programs, typically terminate-and-stay-resident programs inside ("LOADHI" or "LOADHIGH").

Interaction between extended memory, expanded-memory emulation and DOS extenders ended up being regulated by the XMS, Virtual Control Program Interface (VCPI), DOS Protected Mode Interface (DPMI) and DOS Protected Mode Services (DPMS) specifications.

Certain emulation programs, colloquially known as LIMulators, did not rely on motherboard or 80386 features at all. Instead, they reserved 64 KiB of the base RAM for the expanded memory window, where they copied data to and from either extended memory or the hard disk when application programs requested page switches. This was programmatically easy to implement, but performance was low. This technique was offered by AboveDisk from Above Software and by several shareware programs.

It is also possible to emulate EMS by using XMS memory on 286 CPUs using 3rd party utilities like EMM286 (.SYS driver).

Decline

Expanded Memory usage declined in the 1990s. The IBM AT Intel 80286 supported 24 bits of address space (16 MiB) in protected mode, and the 386 supported 32-bit addresses, or 4 gigabytes (2³²) of RAM – 4096 times the addressable space of the original 8086. DOS itself did not directly support protected mode, but Microsoft eventually developed DPMI, and several DOS extenders were published based on it. DOS programs like Doom could use extenders like DOS/4G to run in protected mode while still using the DOS API. In the early 1990s new operating systems like Linux, Windows 9x, Windows NT, OS/2, and BSD/OS supported protected mode "out of the box". These and similar developments rendered Expanded Memory an obsolete concept.

Similar concepts

Other platforms have implemented the same basic concept – additional memory outside of the main address space – but in technically incompatible ways:

• Expanded storage was a feature on IBM mainframes providing additional memory outside of the main system memory, first introduced with the IBM 3090 high-end mainframe series in 1985. Expanded storage could not be directly addressed by applications; an MVS feature known as "window services" enabled applications to allocate movable windows to expanded storage within their own address space. There was also a "data mover" feature which could be invoked to move data between main memory (central storage) and expanded storage; later, an "Asynchronous Data Mover Facility" (ADMF) was introduced, which enabled applications to request data to be moved between the two in the background, while they performed other processing. By the mid-1990s, expanded storage had ceased to be a physically separate memory, and had become merely a logical division within the system memory enforced by firmware; but it was not until the November 2016 release of z/VM 6.4 that IBM finally removed all support for expanded storage from its mainframe operating systems.
• Address Windowing Extensions (AWE) is a conceptually similar feature in Microsoft Windows, used to enable 32-bit applications to access more memory than the 2–4GB that can fit in a 32-bit address space. Although still supported by current versions of Windows, its use has been superseded by 64-bit applications, which can access >4GB of memory directly.
• Virtual memory creates the illusion of available memory using, for instance, disk storage.

See also

• Conventional memory
• DOS memory management
• Extended memory (XMS)
• High memory area (HMA)
• Overlay (programming)
• Upper memory area (UMA)
• Global EMM Import Specification (GEMMIS)
• x86 memory segmentation
• Address Windowing Extensions (AWE)
• Physical Address Extension (PAE)

Further reading

• "Expanded Memory Specification". Lotus, Intel, Microsoft. October 1987. http://www.phatcode.net/res/218/files/limems40.txt. 
• Tischer, Michael (1989). PC System Programming for Developers. Abacus. ISBN 1-55755-036-0. https://archive.org/details/PC_System_Programming. 
• "Understanding EMS 4.0". Technical Cornucopia. January–February 1989. http://www.borrett.id.au/computing/art-1989-01-02.htm.