Pointer swizzling

Short description: Computer science term

In computer science, pointer swizzling is the conversion of references based on name or position into direct pointer references (memory addresses). It is typically performed during deserialization or loading of a relocatable object from a disk file, such as an executable file or pointer-based data structure.

The reverse operation, replacing memory pointers with position-independent symbols or positions, is sometimes referred to as unswizzling, and is performed during serialization (saving).

Example

It is easy to create a linked list data structure using elements like this:

struct node {
        int data;
        struct node *next;
};

But saving the list to a file and then reloading it will (on most operating systems) break every link and render the list useless because the nodes will almost never be loaded into the same memory locations. One way to usefully save and retrieve the list is to assign a unique id number to each node and then unswizzle the pointers by turning them into a field indicating the id number of the next node:

struct node_saved {
        int data;
        int id_number;
        int id_number_of_next_node;
};

Records like these can be saved to a file in any order and reloaded without breaking the list. Other options include saving the file offset of the next node or a number indicating its position in the sequence of saved records, or simply saving the nodes in-order to the file.

After loading such a list, finding a node based on its number is cumbersome and inefficient (serial search). Traversing the list was very fast with the original "next" pointers. To convert the list back to its original form, or swizzle the pointers, requires finding the address of each node and turning the id_number_of_next_node fields back into direct pointers to the right node.

Methods of unswizzling

There are a potentially unlimited number of forms into which a pointer can be unswizzled, but some of the most popular include:

• The offset of the pointed-to object in the file
• The index of the pointed-to object in some sequence of records
• A unique identifier possessed by the pointed-to object, such as a person's Social Security number; in databases, all pointers are unswizzled in this manner (see Foreign key).

Methods of swizzling

Swizzling in the general case can be complicated. The reference graph of pointers might contain an arbitrary number of cycles; this complicates maintaining a mapping from the old unswizzled values to the new addresses. Associative arrays are useful for maintaining the mapping, while algorithms such as breadth-first search help to traverse the graph, although both of these require extra storage. Various serialization libraries provide general swizzling systems. In many cases, however, swizzling can be performed with simplifying assumptions, such as a tree or list structure of references.

The different types of swizzling are:

• Automatic swizzling
• On-demand swizzling

Potential security weaknesses

For security, unswizzling and swizzling must be implemented with great caution. In particular, an attacker's presentation of a specially crafted file may allow access to addresses outside of the expected and proper bounds. In systems with weak memory protection this can lead to exposure of confidential data or modification of code likely to be executed. If the system does not implement guards against execution of data the system may be severely compromised by the installation of various kinds of malware.

Methods of protection include verifications prior to releasing the data to an application:

• That every offset lies within the bounds of the data read.
• That a table of indexes and the records pointed to is similarly constrained.
• That identifiers are unique and, if sensitive, encrypted.
• That all variable-length data is restrained to lengths not exceeding the actual allocation.
• That allocations are of reasonable size.
• That allocations made that are not loaded with data read are cleared, or loaded with some specific pattern.

See also

• Relocation, an eager form of pointer modification

Further reading

• "Pointer swizzling at page fault time: efficiently supporting huge address spaces on standard hardware". ACM SIGARCH Computer Architecture News 19 (4): pp. 6–13. 1991-07-01. doi:10.1145/122576.122577. 
• "Adaptable Pointer Swizzling Strategies in Object Bases: Design, Realization, and Quantitative Analysis". The International Journal on Very Large Data Bases 4 (3): 519–567. July 1995. doi:10.1007/BF01231646. http://www.vldb.org/journal/VLDBJ4/P519.pdf. Retrieved 2021-12-08.  (49 pages)
• Derek's ABC of C. 2. June 1992. pp. 340–343.