Redaction

Short description: Removing sensitive information from a document to allow distribution

Redaction or sanitization is the process of removing sensitive information from a document so that it may be distributed to a broader audience. It is intended to allow the selective disclosure of information. Typically, the result is a document that is suitable for publication or for dissemination to others rather than the intended audience of the original document.

When the intent is secrecy protection, such as in dealing with classified information, redaction attempts to reduce the document's classification level, possibly yielding an unclassified document. When the intent is privacy protection, it is often called data anonymization. Originally, the term sanitization was applied to printed documents; it has since been extended to apply to computer files and the problem of data remanence.

Government secrecy

In the context of government documents, redaction (also called sanitization) generally refers more specifically to the process of removing sensitive or classified information from a document prior to its publication, during declassification.

Secure document redaction techniques

A 1953 US government document that has been redacted prior to release.

A heavily redacted page from a 2004 lawsuit filed by the ACLU — American Civil Liberties Union v. Ashcroft

Redacting confidential material from a paper document before its public release involves overwriting portions of text with a wide black pen, followed by photocopying the result—the obscured text may be recoverable from the original. Alternatively opaque "cover up tape" or "redaction tape", opaque, removable adhesive tape in various widths, may be applied before photocopying.

This is a simple process with only minor security risks. For example, if the black pen or tape is not wide enough, careful examination of the resulting photocopy may still reveal partial information about the text, such as the difference between short and tall letters. The exact length of the removed text also remains recognizable, which may help in guessing plausible wordings for shorter redacted sections. Where computer-generated proportional fonts were used, even more information can leak out of the redacted section in the form of the exact position of nearby visible characters.

The UK National Archives published a document, Redaction Toolkit, Guidelines for the Editing of Exempt Information from Documents Prior to Release, "to provide guidance on the editing of exempt material from information held by public bodies."

Secure redacting is more complicated with computer files. Word processing formats may save a revision history of the edited text that still contains the redacted text. In some file formats, unused portions of memory are saved that may still contain fragments of previous versions of the text. Where text is redacted, in Portable Document (PDF) or word processor formats, by overlaying graphical elements (usually black rectangles) over text, the original text remains in the file and can be uncovered by simply deleting the overlaying graphics. Effective redaction of electronic documents requires the removal of all relevant text and image data from the document file. This process, internally complex, can be carried out very easily by a user with the aid of "redaction" functions in software for editing PDF or other files.

Redaction may administratively require marking of the redacted area with the reason that the content is being restricted. US government documents released under the Freedom of Information Act are marked with exemption codes that denote the reason why the content has been withheld.

The US National Security Agency (NSA) published a guidance document which provides instructions for redacting PDF files.

Printed matter

A page of a classified document that has been sanitized for public release. This is page 13 of a U.S. National Security Agency report [1] on the USS Liberty incident, which was declassified and released to the public in July 2003. Classified information has been blocked out so that only the unclassified information is visible. Notations with leader lines at top and bottom cite statutory authority for not declassifying certain sections. Click on the image to enlarge.

Printed documents which contain classified or sensitive information frequently contain a great deal of information which is less sensitive. There may be a need to release the less sensitive portions to uncleared personnel. The printed document will consequently be sanitized to obscure or remove the sensitive information. Maps have also been redacted for the same reason, with highly sensitive areas covered with a slip of white paper.

In some cases, sanitizing a classified document removes enough information to reduce the classification from a higher level to a lower one. For example, raw intelligence reports may contain highly classified information such as the identities of spies, that is removed before the reports are distributed outside the intelligence agency: the initial report may be classified as Top Secret while the sanitized report may be classified as Secret.

In other cases, such as the NSA report on the USS Liberty incident (right), the report may be sanitized to remove all sensitive data, so that the report may be released to the general public.

As is seen in the USS Liberty report, paper documents are usually sanitized by covering the classified and sensitive portions before photocopying the document.

Computer media and files

Computer (electronic or digital) documents are more difficult to sanitize. In many cases, when information in an information system is modified or erased, some or all of the data remains in storage. This may be an accident of design, where the underlying storage mechanism (disk, RAM, etc.) still allows information to be read, despite its nominal erasure. The general term for this problem is data remanence. In some contexts (notably the US NSA, DoD, and related organizations), "sanitization" typically refers to countering the data remanence problem.

However, the retention may be a deliberate feature, in the form of an undo buffer, revision history, "trash can", backups, or the like. For example, word processing programs like Microsoft Word will sometimes be used to edit out the sensitive information. These products do not always show the user all of the information stored in a file, so it is possible that a file may still contain sensitive information. In other cases, inexperienced users use ineffective methods which fail to sanitize the document. Metadata removal tools are designed to effectively sanitize documents by removing potentially sensitive information.

In May 2005 the US military published a report on the death of Nicola Calipari, an Italian secret agent, at a US military checkpoint in Iraq. The published version of the report was in PDF format, and had been incorrectly redacted by covering sensitive parts with opaque blocks in software. Shortly thereafter, readers discovered that the blocked-out portions could be retrieved by copying and pasting them into a word processor.

On May 24, 2006, lawyers for the communications service provider AT&T filed a legal brief regarding their cooperation with domestic wiretapping by the NSA. Text on pages 12 to 14 of the PDF document were incorrectly redacted, and the covered text could be retrieved.

At the end of 2005, the NSA released a report giving recommendations on how to safely sanitize a Microsoft Word document.

Issues such as these make it difficult to reliably implement multilevel security systems, in which computer users of differing security clearances may share documents. The Challenge of Multilevel Security gives an example of a sanitization failure caused by unexpected behavior in Microsoft Word's change tracking feature.

The two most common mistakes for incorrectly redacting a document are adding an image layer over the sensitive text to obscure it, without removing the underlying text, and setting the background color to match the text color. In both of these cases, the redacted material still exists in the document underneath the visible appearance and is subject to searching and even simple copy and paste extraction. Proper redaction tools and procedures must be used to permanently remove the sensitive information. This is often accomplished in a multi-user workflow where one group of people mark sections of the document as proposals to be redacted, another group verifies the redaction proposals are correct, and a final group operates the redaction tool to permanently remove the proposed items.

See also

• Censorship
• Data erasure
• Data remanence
• Freedom of information laws by country
• Lacuna

External links

• Embarrassing Redaction Failures from Vol. 58 No. 2, May 2019, Technology column of The Judges' Journal published by the American Bar Association