NTFS reparse point

From HandWiki

An NTFS reparse point is a type of NTFS file system object. It is available with the NTFS v3.0 found in Windows 2000 or later versions. Reparse points provide a way to extend the NTFS filesystem. A reparse point contains a reparse tag and data that are interpreted by a filesystem filter driver identified by the tag. Microsoft includes several default tags including NTFS symbolic links, directory junction points, volume mount points and Unix domain sockets. Also, reparse points are used as placeholders for files moved by Windows 2000's Remote Storage Hierarchical Storage System. They also can act as hard links[citation needed], but are not limited to pointing to files on the same volume: they can point to directories on any local volume. The feature[which?] is inherited to ReFS.[1] The open source NTFS-3G driver implements built-in support for the link-type reparse points, namely symbolic links and junction points. A plugin filter system is available to handle additional types of reparse points, allowing for chunk-deduplicated files, system-compressed files, and OneDrive files to be read.[2]

Structure

A reparse point has the following general structure, in C struct form: 

struct REPARSE_BUFFER {
  uint32_t  ReparseTag;
  uint32_t  ReparseDataLength;
  uint16_t  Reserved;
  uint8_t   DataBuffer[]; // flexible array member
}

The reparse tag[3] is unique to each type of reparse point. It defines to which reparse point handler (usually a file system filter driver) the I/O manager delegates the processing.[4] Microsoft provides documentation on some "public" tag types.[5]

Types

Volume mount points

Volume mount points are similar to Unix mount points, where the root of another file system is attached to a directory. In NTFS, this allows additional file systems to be mounted without requiring a separate drive letter (such as C: or D:) for each.

Once a volume has been mounted on top of an existing directory of another volume, the contents previously listed in that directory become invisible and are replaced by the content of the root directory of the mounted volume.[citation needed] The mounted volume could still have its own drive letter assigned separately. The file system does not allow volumes to be mutually mounted on each other. Volume mount points can be made to be either persistent (remounted automatically after system reboot) or not persistent (must be manually remounted after reboot).[citation needed]

Mounted volumes may use other file systems than just NTFS, possibly with their own security settings and remapping of access rights according to the remote file system policy.

The substitute names of volume mount points use the NT namespace form \??\DeviceName\.[6][7][4] Junctions generally use \??\<drive>:\ to refer to a volume with an existing driver letter, while true volume mount points use \??\Volume{<guid>} to refer to any volume. UNC paths are invalid for junctions.[8]

Directory junctions

Main page: NTFS junction point

Directory junctions are defined using the exact same mechanism (and reparse tag: IO_REPARSE_TAG_MOUNT_POINT) as volume mount points are. The only difference is that their substitute names point to a subdirectory of another volume that usually already has a drive letter. This function is conceptually similar to symbolic links to directories in Unix, except that the target in NTFS must always be another directory (typical Unix file systems allow the target of a symbolic link to be any type of file).[4]

For instance, the directory C:\exampledir with a directory junction attribute that contains a link to D:\linkeddir will automatically refer to the directory D:\linkeddir when it is accessed by a user-mode application.[9]

Directory junctions (which can be created with the command MKLINK /J junctionName targetDirectory and removed with RMDIR junctionName from a console prompt) are persistent, and resolved on the server side as they share the same security realm of the local system or domain on which the parent volume is mounted and the same security settings for its contents as the content of the target directory; however the junction itself may have distinct security settings. Unlinking a directory junction does not delete files in the target directory.

Some directory junctions are installed by default on Windows Vista, for compatibility with previous versions of Windows, such as Documents and Settings in the root directory of the system drive, which links to the Users physical directory in the root directory of the same volume. However they are hidden by default, and their security settings are set up so that the Windows Explorer will refuse to open them from within the Shell or in most applications, except for the local built-in SYSTEM user or the local Administrators group (both user accounts are used by system software installers). This additional security restriction has probably been made to avoid users of finding apparent duplicate files in the joined directories and deleting them by error, because the semantics of directory junctions are not the same as for hard links; the reference counting is not used on the target contents and not even on the referenced container itself.[citation needed]

Directory junctions are soft links (they will persist even if the target directory is removed), working as a limited form of symbolic links (with an additional restriction on the location of the target), but it is an optimized version allowing faster processing of the reparse point with which they are implemented, with less overhead than the newer NTFS symbolic links, and can be resolved on the server side (when they are found in remote shared directories).[citation needed]

Symbolic links

Main page: NTFS symbolic link

Symbolic links (or soft links) were introduced in Windows Vista.[10] Symbolic links are resolved on the client side. So when a symbolic link is shared, the target is subject to the access restrictions on the client, and not the server.[citation needed]

Symbolic links can be created either to files (created with MKLINK symLink targetFilename) or to directories (created with MKLINK /D symLinkD targetDirectory), but (unlike Unix symbolic links) the semantic of the link must be provided with the created link. The target however need not exist or be available when the symbolic link is created: when the symbolic link will be accessed and the target will be checked for availability, NTFS will also check if it has the correct type (file or directory); it will return a not-found error if the existing target has the wrong type.[citation needed]

They can also reference shared directories on remote hosts or files and subdirectories within shared directories: their target is not mounted immediately at boot, but only temporarily on demand while opening them with the OpenFile() or CreateFile() API. Their definition is persistent on the NTFS volume where they are created (all types of symbolic links can be removed as if they were files, using DEL symLink from a command line prompt or batch).[citation needed]

The symbolic link data is similar to mount point data, in that both use an NT namespace path. The difference is that symbolic links accepts UNC paths, but not Volume{guid} mounts.[8]

Distributed Link Tracking (DLT)

Distributed link tracking allows applications to track files, shell shortcuts or OLE links even if they were renamed or moved to another volume within the same machine, domain or workgroup.[11] Tracking is implemented as a system service, which uses the object identifier (OID) index stored in a metafile.[12] When the application requests a track to a file or directory, the tracking service creates the OID entry, which points to the file, and file rename, copy or move operation to a NTFS v3 volume also copies the object ID. This allows the tracking service to eventually find the target file.

Data deduplication

When there are several directories that have different but similar files, some of these files may have identical content. Single instance storage, found in Windows Server 2000 through Windows Storage Server 2008, allows identical files to be merged to one file and create references to that merged file. SIS consists of a file system filter that manages copies, modification and merges to files; and a user space service (or groveler) that searches for files that are identical and need merging. SIS was mainly designed for remote installation servers as these may have multiple installation images that contain many identical files; SIS allows these to be consolidated but, unlike for example hard links, each file remains distinct; changes to one copy of a file will leave others unaltered. This is similar to copy-on-write, which is a technique by which memory copying is not really done until one copy is modified.[13]

Since Windows Server 2012, there is a new chunk-based data deduplication mechanism (tag 0x80000013) that allows files with similar content to be deduplicated as long as they have stretches of identical data.[2] This mechanism is more powerful than SIS.[14] Since Windows Server 2019, the feature is fully supported on ReFS.[15]

Hierarchical Storage Management (HSM)

Hierarchical Storage Management is a means of transferring files that are not used for some period of time to less expensive storage media. When the file is next accessed, the reparse point on that file determines that it is needed and retrieves it from storage.[citation needed]

Native Structured Storage (NSS)

NSS was an ActiveX document storage technology that has since been discontinued by Microsoft.[citation needed] It allowed ActiveX Documents to be stored in the same multi-stream format that ActiveX uses internally. An NSS file system filter was loaded and used to process the multiple streams transparently to the application, and when the file was transferred to a non-NTFS formatted disk volume it would also transfer the multiple streams into a single stream.[16]

Unix domain socket (socket)

With Windows 10 build 17063 (for stable release version 1803), Microsoft introduced Unix domain sockets to Windows. These are realized by using the afunix.sys kernel driver and a new reparse point in the file system. Unix domain sockets are common on BSD and Linux systems and can be seen as the standard for inter process communication on these systems; therefore their introduction to Windows will allow simplified adoption of code and cross-platform portability.[17]

System compression

Windows 10 introduces read-only CompactOS compression algorithms for NTFS file system, taken from Windows Imaging Format (WIM); they are designed to compress Windows system files and reduce disk space usage.[18]

Internally, the compressed file is recorded as a reparse point with tag IO_REPARSE_TAG_WOF (0x80000017), where WoF stands for Windows Overlay Filter,[19] and the actual data is stored in an alternate data stream named "WofCompressedData", which is processed by WOF file system filter.[20][21][2]

CompactOS is an improved variant of WIMBoot from Windows 8.1, where system files could be stored in a compressed WIM image on a hidden disk partition,[22] and the WOF filter driver would decompress file contents from that WIM file; using alternate data streams instead of read-only WIM images allows CompactOS to recompress system files when they need to be rewritten with an updated version.[23]

OneDrive

OneDrive tags the files and directories it has downloaded to the local storage as a reparse point with tag 0x9000001a. The actual data is stored normally.[2]

Known risks

The Stuxnet as part of its series of Win32 exploits uses NTFS junction points as part of its overall mode of operation.[citation needed]

See also

References

  1. ↑ "Microsoft Windows Vista Client Configuration Study Guide" Wiley Publishing, Inc. 2007 p.285
  2. ↑ 2.0 2.1 2.2 2.3 André, Jean-Pierre (March 1, 2019). "NTFS-3G: Junction Points, Symbolic Links and Reparse Points". https://github.com/tuxera/ntfs-3g/wiki/Junctions-Points,-Symbolic-Links-and-Reparse-Points#other-types-of-reparse-points. 
  3. ↑ "Reparse Point Tags". https://docs.microsoft.com/en-us/windows/win32/fileio/reparse-point-tags. 
  4. ↑ 4.0 4.1 4.2 "NTFS Links, Directory Junctions, and Windows Shortcuts". http://www.flexhex.com/docs/articles/hard-links.phtml. 
  5. ↑ "[MS-FSCC Reparse Tags"]. https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/c8e77b37-3909-4fe6-a4ea-2b9d423b1ee4. 
  6. ↑ "Naming Files, Paths, and Namespaces/NT Namespaces". https://docs.microsoft.com/en-us/windows/win32/fileio/naming-a-file#nt-namespaces. 
  7. ↑ "winapi - Does the "SubstituteName" string in the PathBuffer of a REPARSE_DATA_BUFFER structure always start with the prefix "\??\", and if so, why?". https://stackoverflow.com/a/14330986. 
  8. ↑ Mark Russinovich. "Inside Win2K NTFS, Part 1". Microsoft Developer Network. http://msdn2.microsoft.com/en-us/library/ms995846.aspx. 
  9. ↑ "Symbolic Links (Windows)". MSDN. http://msdn.microsoft.com/en-us/library/aa365680(VS.85).aspx. 
  10. ↑ "Distributed Link Tracking and Object Identifiers". http://msdn.microsoft.com/en-us/library/windows/desktop/aa363997.aspx. 
  11. ↑ "Distributed Link Tracking Client (System Services for the Windows Server 2003 Family and Windows XP Operating Systems)". https://technet.microsoft.com/en-us/library/cc736811(WS.10).aspx. 
  12. ↑ Bolosky, Bill; Corbin, Scott; Goebel, David; Douceur, John (January 2000). "Single Instance Storage in Windows 2000" (PDF). Proceedings of 4th USENIX Windows Systems Symposium. Seattle, Washington: Microsoft Research and Balder Technology Group. http://research.microsoft.com/apps/pubs/default.aspx?id=74261. 
  13. ↑ "Introduction to Data Deduplication in Windows Server 2012" (in en). 10 April 2019. https://techcommunity.microsoft.com/t5/Storage-at-Microsoft/Introduction-to-Data-Deduplication-in-Windows-Server-2012/ba-p/424257. 
  14. ↑ "Data Deduplication interoperability" (in en-us). https://docs.microsoft.com/en-us/windows-server/storage/data-deduplication/interop. 
  15. ↑ Saville, John (date unknown). What is Native Structured Storage? Windows IT Pro. Retrieved from "What is Native Structured Storage?". http://www.windowsitpro.com/Article/ArticleID/13785/13785.html. .
  16. ↑ "AF_UNIX comes to Windows" (in en-US). Windows Command Line Tools For Developers. https://blogs.msdn.microsoft.com/commandline/2017/12/19/af_unix-comes-to-windows/. 
  17. ↑ "Compact OS, single-instancing, and image optimization" (in en-us). Microsoft. https://docs.microsoft.com/en-us/windows-hardware/manufacture/desktop/compact-os. 
  18. ↑ "What is WofCompressedData? Does WOF mean that Windows is a dog?". 18 June 2019. https://devblogs.microsoft.com/oldnewthing/20190618-00/?p=102597. 
  19. ↑ "Re: [ntfs-3g-devel Experimental support for Windows 10 "System Compressed" files"]. https://sourceforge.net/p/ntfs-3g/mailman/message/34481588/. 
  20. ↑ Biggers, Eric (29 April 2019). "ntfs-3g-system-compression: NTFS-3G plugin for reading "system compressed" files". https://github.com/ebiggers/ntfs-3g-system-compression. 
  21. ↑ Windows Image File Boot (WIMBoot) Overview
  22. ↑ Raymond Chen.What is WofCompressedData? Does WOF mean that Windows is a dog? Microsoft DevBlogs.

External links



Retrieved from "https://handwiki.org/wiki/index.php?title=NTFS_reparse_point&oldid=3378351"