Link rot (also called link death, link breaking, or reference rot) is the phenomenon of hyperlinks tending over time to cease to point to their originally targeted file, web page, or server due to that resource being relocated to a new address or becoming permanently unavailable. A link that no longer points to its target, often called a broken, dead, or orphaned link, is a specific form of dangling pointer.
The rate of link rot is a subject of study and research due to its significance to the internet's ability to preserve information. Estimates of that rate vary dramatically between studies. Information professionals have warned that link rot could make important archival data disappear, potentially impacting the legal system and scholarship.
A 2003 study found that on the Web, about one link out of every 200 broke each week, suggesting a half-life of 138 weeks. This rate was largely confirmed by a 2016–2017 study of links in Yahoo! Directory (which had stopped updating in 2014 after 21 years of development) that found the half-life of the directory's links to be two years.
A 2004 study showed that subsets of Web links (such as those targeting specific file types or those hosted by academic institution) could have dramatically different half-lives. The URLs selected for publication appear to have greater longevity than the average URL. A 2015 study by Weblock analyzed more than 180,000 links from references in the full-text corpora of three major open access publishers and found a half-life of about 14 years, generally confirming a 2005 study that found that half of the URLs cited in D-Lib Magazine articles were active 10 years after publication. Other studies have found higher rates of link rot in academic literature but typically suggest a half-life of four years or greater. A 2013 study in BMC Bioinformatics analyzed nearly 15,000 links in abstracts from Thomson Reuters's Web of Science citation index and found that the median lifespan of web pages was 9.3 years, and just 62% were archived. A 2021 study of external links in New York Times articles published between 1996 and 2019 found a half-life of about 15 years (with significant variance among content topics) but noted that 13% of functional links no longer lead to the original content—a phenomenon called content drift.
A 2002 study suggested that link rot within digital libraries is considerably slower than on the web, finding that about 3% of the objects were no longer accessible after one year (equating to a half-life of nearly 23 years).
Link rot can result from several occurrences. A target web page may be removed. The server that hosts the target page could fail, be removed from service, or relocate to a new domain name. A domain name's registration may lapse or be transferred to another party. Some causes will result in the link failing to find any target and returning an error such as HTTP 404. Other causes will cause a link to target content other than what was intended by the link's author.
Other reasons for broken links include:
- the restructuring of websites that causes changes in URLs (e.g.
domain.net/pine_treemight be moved to
- relocation of formerly free content to behind a paywall
- a change in server architecture that results in code such as PHP functioning differently
- dynamic page content such as search results that changes by design
- deletion of the target page and/or its content
- the presence of user-specific information (such as a login name) within the link
- deliberate blocking by content filters or firewalls
- the expiration of a domain name registration
Prevention and detection
Strategies for preventing link rot can focus on placing content where its likelihood of persisting is higher, authoring links that are less likely to be broken, taking steps to preserve existing links, or repairing links whose targets have been relocated or removed.
The creation of URLs that will not change with time is the fundamental method of preventing link rot. Preventive planning has been championed by Tim Berners-Lee and other web pioneers.
Strategies pertaining to the authorship of links include:
- linking to primary rather than secondary sources and prioritizing stable sites
- avoiding links that point to resources on researchers' personal pages
- using clean URLs or otherwise employing URL normalization or URL canonicalization
- using permalinks and persistent identifiers such as ARKs, DOIs, Handle System references, PURLs, or content addressing
- avoiding linking to documents other than web pages
- avoiding deep linking
- linking to web archives such as the Internet Archive, WebCite, archive.today, Perma.cc, Amber or Arweave
Strategies pertaining to the protection of existing links include:
- using redirection mechanisms such as HTTP 301 to automatically refer browsers and crawlers to relocated content.
- using content management systems which can automatically update links when content within the same site is relocated or automatically replace links with canonical URLs
- integrating search resources into HTTP 404 pages
The detection of broken links may be done manually or automatically. Automated methods include plug-ins for content management systems as well as standalone broken-link checkers such as like Xenu's Link Sleuth. Automatic checking may not detect links that return a soft 404 or links that return a 200 OK response but point to content that has changed.
- Software rot
- Digital preservation
- Deletionism and inclusionism in Wikipedia
- Archive Team, web archiving team
- Markwell, John; Brooks, David W. (2002). "Broken Links: The Ephemeral Nature of Educational WWW Hyperlinks". Journal of Science Education and Technology 11 (2): 105–108. doi:10.1023/A:1014627511641.
- Gomes, Daniel; Silva, Mário J. (2006). "Modelling Information Persistence on the Web". ICWE'06. http://xldb.di.fc.ul.pt/daniel/docs/papers/gomes06urlPersistence.pdf. Retrieved 14 September 2010.
- Dellavalle, Robert P.; Hester, Eric J.; Heilig, Lauren F.; Drake, Amanda L.; Kuntzman, Jeff W.; Graber, Marla; Schilling, Lisa M. (2003). "Going, Going, Gone: Lost Internet References". Science 302 (5646): 787–788. doi:10.1126/science.1088234. PMID 14593153. https://www.researchgate.net/publication/9030760.
- Koehler, Wallace (1999). "An Analysis of Web Page and Web Site Constancy and Permanence". Journal of the American Society for Information Science 50 (2): 162–180. doi:10.1002/(SICI)1097-4571(1999)50:2<162::AID-ASI7>3.0.CO;2-B.
- Sellitto, Carmine (2005). "The impact of impermanent Web-located citations: A study of 123 scholarly conference publications". Journal of the American Society for Information Science and Technology 56 (7): 695–703. doi:10.1002/asi.20159. http://vuir.vu.edu.au/336/1/The_impact_of_impermanent_Web.pdf.
- Fetterly, Dennis; Manasse, Mark; Najork, Marc; Wiener, Janet (2003). "A large-scale study of the evolution of web pages". http://www2003.org/cdrom/papers/refereed/p097/P97%20sources/p97-fetterly.html. Retrieved 14 September 2010.
- van der Graaf, Hans. "The half-life of a link is two year". http://blog.zomdir.com/2017/10/the-half-life-of-link-is-two-year.html.
- Koehler, Wallace (2004). "A longitudinal study of web pages continued: a consideration of document persistence". Information Research 9 (2). http://www.informationr.net/ir/9-2/paper174.html. Retrieved 2019-01-31.
- "All-Time Weblock Report". August 2015. https://weblock.io/report?id=all.
- McCown, Frank; Chan, Sheffan; Nelson, Michael L.; Bollen, Johan (2005). "The Availability and Persistence of Web References in D-Lib Magazine". http://www.iwaw.net/05/papers/iwaw05-mccown1.pdf. Retrieved 2005-10-12.
- Spinellis, Diomidis (2003). "The Decay and Failures of Web References". Communications of the ACM 46 (1): 71–77. doi:10.1145/602421.602422. http://www.spinellis.gr/pubs/jrnl/2003-CACM-URLcite/html/urlcite.html. Retrieved 2007-09-29.
- , Wikidata Q21012586
- Hennessey, Jason; Xijin Ge, Steven (2013). "A Cross Disciplinary Study of Link Decay and the Effectiveness of Mitigation Techniques". BMC Bioinformatics 14 (Suppl 14): S5. doi:10.1186/1471-2105-14-S14-S5. PMID 24266891.
- "What the ephemerality of the Web means for your hyperlinks" (in en). https://www.cjr.org/analysis/linkrot-content-drift-new-york-times.php.
- Nelson, Michael L.; Allen, B. Danette (2002). "Object Persistence and Availability in Digital Libraries". D-Lib Magazine 8 (1). doi:10.1045/january2002-nelson. https://digitalcommons.odu.edu/cgi/viewcontent.cgi?article=1008&context=computerscience_fac_pubs. Retrieved 2019-09-24.
- Berners-Lee, Tim (1998). "Cool URIs Don't Change". https://www.w3.org/Provider/Style/URI.
- Kille, Leighton Walter (8 November 2014). "The Growing Problem of Internet "Link Rot" and Best Practices for Media and Online Publishers". Journalist's Resource, Harvard Kennedy School. http://journalistsresource.org/studies/society/internet/website-linking-best-practices-media-online-publishers.
- Sicilia, Miguel-Angel, et al. "Decentralized Persistent Identifiers: a basic model for immutable handlers." Procedia computer science 146 (2019): 123-130.
- "Internet Archive: Digital Library of Free Books, Movies, Music & Wayback Machine". 2001-03-10. https://archive.org/.
- Eysenbach, Gunther; Trudel, Mathieu (2005). "Going, going, still there: Using the WebCite service to permanently archive cited web pages". Journal of Medical Internet Research 7 (5): e60. doi:10.2196/jmir.7.5.e60. PMID 16403724.
- Zittrain, Jonathan; Albert, Kendra; Lessig, Lawrence (12 June 2014). "Perma: Scoping and Addressing the Problem of Link and Reference Rot in Legal Citations". Legal Information Management 14 (2): 88–99. doi:10.1017/S1472669614000255. https://cdn.harvardlawreview.org/wp-content/uploads/2014/03/forvol127_zittrain.pdf. Retrieved 10 June 2020.
- "Harvard University's Berkman Center Releases Amber, a "Mutual Aid" Tool for Bloggers & Website Owners to Help Keep the Web Available | Berkman Center". https://cyber.law.harvard.edu/node/99276.
- "Arweave - A community-driven ecosystem". https://arweave.org/.
- Rønn-Jensen, Jesper (2007-10-05). "Software Eliminates User Errors And Linkrot". Justaddwater.dk. http://justaddwater.dk/2007/10/05/blog-software-eliminates-user-errors-and-linkrot/.
- Mueller, John (2007-12-14). "FYI on Google Toolbar's Latest Features". Google Webmaster Central Blog. http://googlewebmastercentral.blogspot.com/2007/12/fyi-on-google-toolbars-latest-features.html.
- Bar-Yossef, Ziv; Broder, Andrei Z.; Kumar, Ravi; Tomkins, Andrew (2004). "Sic transit gloria telae: towards an understanding of the Web's decay". pp. 328–337. doi:10.1145/988672.988716. ISBN 978-1581138443.
|The Wikibook Authoring Webpages has a page on the topic of: Preventing link rot|
- Future-Proofing Your URIs
- Nielsen, Jakob (14 June 1998). "Fighting Linkrot". http://www.useit.com/alertbox/980614.html.
Original source: https://en.wikipedia.org/wiki/Link rot. Read more