Open-source intelligence

From HandWiki
Revision as of 14:00, 6 February 2024 by Steve2012 (talk | contribs) (url)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Short description: Data collected from publicly available sources to be used in an intelligence context

Open-source intelligence (OSINT) is the collection and analysis of data gathered from open sources (covert sources and publicly available information; PAI) to produce actionable intelligence. OSINT is primarily used in national security, law enforcement, and business intelligence functions and is of value to analysts who use non-sensitive intelligence in answering classified, unclassified, or proprietary intelligence requirements across the previous intelligence disciplines.[1]

Categories

OSINT sources can be divided up into six different categories of information flow:[2]

  • Media, print newspapers, magazines, radio, and television from across and between countries.
  • Internet, online publications, blogs, discussion groups, citizen media (i.e. – cell phone videos, and user created content), YouTube, and other social media websites (i.e. – Facebook, Twitter, Instagram, etc.). This source also outpaces a variety of other sources due to its timeliness and ease of access.
  • Public government data, public government reports, budgets, hearings, telephone directories, press conferences, websites, and speeches. Although this source comes from an official source they are publicly accessible and may be used openly and freely.
  • Professional and academic publications, information acquired from journals, conferences, symposia, academic papers, dissertations, and theses.
  • Commercial data, commercial imagery, financial and industrial assessments, and databases.
  • Grey literature, technical reports, preprints, patents, working papers, business documents, unpublished works, and newsletters.

OSINT is distinguished from research in that it applies the process of intelligence to create tailored knowledge supportive of a specific decision by a specific individual or group.[3]

OSINT collection methodologies

Collecting open-source intelligence is achieved in a variety of different ways,[4] such as:

  • Social Media Intelligence, which is acquired from viewing or observing a subjects online social profile activity.
  • Search engine data mining or scraping.
  • Public records checking.
  • Information matching and verification from data broker services.

Definition

OSINT is defined in the United States of America by Public Law 109-163 as cited by both the U.S. Director of National Intelligence and the U.S. Department of Defense (DoD), as intelligence "produced from publicly available information that is collected, exploited, and disseminated in a timely manner to an appropriate audience for the purpose of addressing a specific intelligence requirement."[5] As defined by NATO, OSINT is intelligence "derived from publicly available information, as well as other unclassified information that has limited public distribution or access."[6]

According to political scientist Jeffrey T. Richelson, “open source acquisition involves procuring verbal, written, or electronically transmitted material that can be obtained legally. In addition to documents and videos available via the Internet or provided by a human source, others are obtained after U.S. or allied forces have taken control of a facility or site formerly operated by a foreign government or terrorist group.”[7]

Former Assistant Director of Central Intelligence for Analysis Mark M. Lowenthal defines OSINT as “any and all information that can be derived from overt collection: all types of media, government reports and other documents, scientific research and reports, commercial vendors of information, the Internet, and so on. The main qualifiers to open-source information are that it does not require any type of clandestine collection techniques to obtain it and that it must be obtained through means that entirely meet the copyright and commercial requirements of the vendors where applicable."[8]

History

Seal of the 9/11 Commission

OSINT practices have been documented as early as the mid-19th century in the United States and early 20th century in the United Kingdom.[9]

OSINT in the United States traces its origins to the 1941 creation of the Foreign Broadcast Monitoring Service (FBMS), an agency responsible for the monitoring of foreign broadcasts. An example of their work was the correlation of changes in the price of oranges in Paris with successful bombings of railway bridges during World War II.[10]

The Aspin-Brown Commission stated in 1996 that US access to open sources was "severely deficient" and that this should be a "top priority" for both funding and DCI attention.[11]

In July 2004, following the September 11 attacks, the 9/11 Commission recommended the creation of an open-source intelligence agency.[12] In March 2005, the Iraq Intelligence Commission recommended[13] the creation of an open-source directorate at the CIA.

Following these recommendations, in November 2005 the Director of National Intelligence announced the creation of the DNI Open Source Center. The Center was established to collect information available from "the Internet, databases, press, radio, television, video, geospatial data, photos and commercial imagery."[14] In addition to collecting openly available information, it would train analysts to make better use of this information. The center absorbed the CIA's previously existing Foreign Broadcast Information Service (FBIS), originally established in 1941, with FBIS head Douglas Naquin named as director of the center.[15] Then, following the events of 9/11 the Intelligence Reform and Terrorism Prevention Act merged FBIS and other research elements into the Office of the Director of National Intelligence creating the Open Source Enterprise.

Furthermore, the private sector has invested in tools which aid in OSINT collection and analysis. Specifically, In-Q-Tel, a Central Intelligence Agency supported venture capital firm in Arlington, VA assisted companies develop web-monitoring and predictive analysis tools.

In December 2005, the Director of National Intelligence appointed Eliot A. Jardines as the Assistant Deputy Director of National Intelligence for Open Source to serve as the Intelligence Community's senior intelligence officer for open source and to provide strategy, guidance and oversight for the National Open Source Enterprise.[16] Mr. Jardines has established the National Open Source Enterprise[17] and authored intelligence community directive 301. In 2008, Mr. Jardines returned to the private sector and was succeeded by Dan Butler who is ADDNI/OS[18] and previously Mr. Jardines' Senior Advisor for Policy.[19]

Tools

It is possible for any user to extract the first and last name provided at sign-up with Kik Messenger, even without user profiles.
Systematic Software Engineering uses open source intelligence, compiled by Janes Information Services, as part of its intelligence platform.
Open source intelligence may be ingested to battle management systems such as CPCE by Systematic, which uses an open source feed from Janes Information Services.

The web browser is a powerful OSINT tool that provides access to numerous websites and both open source and proprietary software tools that are either purpose-built for open source information collection or which can be exploited for the purposes of either gathering of open source information or to facilitate analysis and validation to provide intelligence. A cottage industry of both for-profit and not-for-profit investigative and educational groups such as Bellingcat, IntelTechniques SANS and others offer indices, books, podcasts and video training materials on OSINT tools and techniques. Books such as Michael Bazzell's Open Source Intelligence Techniques serve as indices to resources across multiple domains but according the author, due to the rapidly changing information landscape, some tools and techniques change or become obsolete frequently, hence it is imperative for OSINT researchers to study, train and survey the landscape of source material regularly.[20] A guide by Ryan Fedasiuk, an analyst at the Center for Security and Emerging Technology, lists six tools open-source analysts can use to stay safe and utilize operational security (OPSEC) when conducting online investigations. These include VPNs, cached webpages, digital archive services, URL and file scanners, browser sandbox applications, and antivirus software.[21]

Numerous lists of aggregated OSINT content are available on the web. The OSINT Framework contains over 30 primary categories of tools and is maintained as an open source project on GitHub.[22]

Risks for practitioners

A main hindrance to practical OSINT is the volume of information it has to deal with ("information explosion"). The amount of data being distributed increases at a rate that it becomes difficult to evaluate sources in intelligence analysis. To a small degree the work has sometimes been done by amateur crowd-sourcing.[23]

Accredited journalists have some protection in asking questions, and researching for recognized media outlets. Even so, they can be imprisoned, even executed, for seeking out OSINT. Private individuals illegally collecting data for a foreign military or intelligence agency is considered espionage in most countries. Of course, espionage that is not treason (i.e. betraying one's country of citizenship) has been a tool of statecraft since ancient times.[24]

Professional association

The OSINT Foundation is a professional association for OSINT practitioners in the United States Intelligence Community.[25] It is open to U.S. Citizens and seeks to raise the prominence of the open-source intelligence discipline.[26]

See also

  • Ashley Feinberg
  • Bellingcat
  • Co-occurrence networks
  • Dan Butler (civil servant)
  • DARPA TIDES program
  • Doxing
  • Eliot A. Jardines
  • Eliot Higgins
  • Fusion center
  • ICWatch
  • Intellipedia
  • Investigative Data Warehouse
  • MiTAP
  • National Intelligence Open Source Committee
  • NATO Open Source Intelligence Handbook, NATO Open Source Intelligence Reader
  • Open data
  • Open Source Center
  • Oryx (blog)
  • Private intelligence agency
  • Social cloud computing
  • Special Libraries Association
  • Strategic intelligence
  • Open-source intelligence in the 2022 Russian invasion of Ukraine


References

  1. ↑ Schwartz, Leo (March 7, 2022). "Amateur open-source researchers went viral unpacking the war in Ukraine". https://restofworld.org/2022/osint-viral-ukraine/. 
  2. ↑ Richelson, Jeffrey (2016). The US Intelligence Community. Avalon. ISBN 978-0813349183. 
  3. ↑ "Spy Agencies Turn to Newspapers, NPR, and Wikipedia for Information: The intelligence community is learning to value 'open-source' information". https://www.usnews.com/articles/news/national/2008/09/12/spy-agencies-turn-to-newspapers-npr-and-wikipedia-for-information.html. 
  4. ↑ Leos, Devan (2023-02-28). "Thinking Like a Spy: How Open Source Intelligence Can Give You a Competitive Advantage" (in en). https://www.entrepreneur.com/growing-a-business/thinking-like-a-spy-how-open-source-intelligence-can-give/444634. 
  5. ↑ "As defined in Sec. 931 of Public Law 109-163, entitled, "National Defense Authorization Act for Fiscal Year 2006."". http://frwebgate.access.gpo.gov/cgi-bin/getdoc.cgi?dbname=109_cong_public_laws&docid=f:publ163.109. 
  6. ↑ "NATOTermOTAN". https://nso.nato.int/natoterm/content/nato/pages/home.html. 
  7. ↑ Richelson, Jeffrey T (2015-07-14) (in en). The U.S. Intelligence Community. Avalon Publishing. ISBN 9780813349190. https://books.google.com/books?id=4sxVDgAAQBAJ&pg=PT284. Retrieved 15 May 2017. 
  8. ↑ Lowenthal, Mark M. (2005), "Open-Source Intelligence: New Myths, New Realities", in George, Roger Z; Kline, Robert D (in en), Intelligence and the national security strategist : enduring issues and challenges, Lanham: Rowman and Littlefield, ISBN 9780742540392, https://books.google.com/books?id=jJxyIb2hN4MC&pg=PA273 
  9. ↑ Block, Ludo (2023). "The long history of OSINT" (in en). Journal of Intelligence History: 1–15. doi:10.1080/16161262.2023.2224091. ISSN 1616-1262. 
  10. ↑ Bornn, D Marshall (9 Jan 2013). "Service members, civilians learn to harness power of 'Open Source' information" (in en). https://www.army.mil/article/94007/Service_members__civilians_learn_to_harness_power_of__Open_Source__information. 
  11. ↑ Lowenthal, Mark; Clark, Robert (2015). The Five Disciplines of Intelligence Collection. CQ Press. p. 18. ISBN 978-1483381114. 
  12. ↑ See page 413 of the 9-11 Commission Report (pdf) .
  13. ↑ McLaughlin, Michael (June 2012). "Using open source intelligence for cybersecurity intelligence" (in en-GB). ComputerWeekly.com. https://www.computerweekly.com/tip/Using-open-source-intelligence-software-for-cybersecurity-intelligence. 
  14. ↑ Office of the Director of National Intelligence. "ODNI Announces Establishment of Open Source Center ". Press release, 8 November 2005.
  15. ↑ Ensor, David. "The Situation Report: Open source intelligence center ". CNN, 8 November 2005.
  16. ↑ Office of the Director of National Intelligence "ODNI Senior Leadership Announcement ". Press release, 7 December 2005.
  17. ↑ "National Open Source Entreprise Vision Statement" May 2006
  18. ↑ DNI Open Source Conference 2008 "Decision Advantage" agenda, Office of the Director of National Intelligence, July 2008.
  19. ↑ DNI Open Source Conference 2007 "Expanding the Horizons" agenda, Office of the Director of National Intelligence, July 2007.
  20. ↑ "Books by Michael Bazzell". https://inteltechniques.com/book1.html. 
  21. ↑ Fedasiuk, Ryan (2022-04-06). "Into the Jungle: Best Practices for Open-Source Researchers" (in en-US). https://cset.georgetown.edu/article/into-the-jungle-best-practices-for-open-source-researchers/. 
  22. ↑ OSINT Framework, lockFALE, 2022-05-02, https://github.com/lockfale/OSINT-Framework, retrieved 2022-05-02 
  23. ↑ "Bellingcat's Eliot Higgins Explains Why Ukraine Is Winning the Information War" (in en). Time. https://time.com/6155869/bellingcat-eliot-higgins-ukraine-open-source-intelligence/. Retrieved 2022-04-13. 
  24. ↑ Sun Tzu (Warring States period), The Art of War, Chapter 13: "Hostile armies may face each other for years, striving for the victory which is decided in a single day. This being so, to remain in ignorance of the enemy's condition simply because one grudges the outlay of 2 hundred ounces of silver in honors and emoluments, is the height of inhumanity."
  25. ↑ (in en-US) New OSINT foundation aims to 'professionalize' open source discipline across spy agencies, 2022-07-27, https://federalnewsnetwork.com/inside-ic/2022/07/new-osint-foundation-aims-to-professionalize-open-source-discipline-across-spy-agencies/ 
  26. ↑ Volz, Dustin (2022-07-27), "New Group to Promote Open-Source Intelligence, Seen as Vital in Ukraine War" (in en-US), Wall Street Journal, https://www.wsj.com/articles/new-group-to-promote-open-source-intelligence-seen-as-vital-in-ukraine-war-11658926800?st=q5j3a3ymqekavuj&reflink=desktopwebshare_permalink 


Literature

Scientific Publications

External links



Retrieved from "https://handwiki.org/wiki/index.php?title=Open-source_intelligence&oldid=3358146"