The Rise and Importance of Open-Source Intelligence (OSINT)
OSINT is defined as intelligence produced by collecting, evaluating and analyzing publicly available information with the purpose of answering specific intelligence questions, has been around for a long time. OSINT in the United States traces its origins to the 1941 creation of the Foreign Broadcast Monitoring Service (FBMS), an agency responsible for the monitoring of foreign broadcasts of our adversaries during WWII.
Driven by the explosion of new forms of publicly available data (social media, Google earth, etc.) and advances in AI/ML OSINT can by some studies today, account for up to 80% to 90% of all intelligence activities carried out by Western Law enforcement and nation agencies. (Open source intelligence and AI: a systematic review of the GELSI literature - PMC (nih.gov))
OSINT sources of information can include:
Public Records
News Media
Websites
Social Media Platforms
Images, Videos
The Dark web
According to market analysis the space is large, around $7B today, and is expected to grow at 28% CAGR annually thru 2030 when the market will be approximately $36B. Global Open Source Intelligence Market Size $36,241.2Mn 2030 (custommarketinsights.com) Law Enforcement followed by Government Intelligence Agencies and Defense Intelligence Agencies are by far the largest customers for OSINT technologies.
US Intelligence Community in its 2023 National Intelligence Strategy calls for agencies to harness open source data alongside other high priority areas like artificial intelligence and advanced analytics. The strategy specifically states that "agencies must continue to build open source intelligence capabilities and augment capacity; enhance sharing of intelligence capabilities; and strengthen the analytical and collection capabilities relating to non- military threats including technology competition"
US DoD through its Defense Innovation Unit (DIU), which is charged with facilitating development and acquisition of cutting-edge technologies at the Defense Department, allocated 10% of its overall FY 23 budget to AI- and ML-based OSINT technologies. (Spotlight on innovation in the 2023 defense policy bill - Washington Technology)
DARPA for FY 24 requested $18 million for SemaFor development. The Semantic Forensics (SemaFor) program seeks to develop innovative semantic technologies for analyzing media and other Open Sources of information.
With agencies relying on OSINT for such a large portion of their Intelligence activities many are grappling with how to improve their ability to leverage this critical capability. An article published in the Modern War Institute Find It, Vet It, Share It: The US Government’s Open-Source Intelligence Problem and How to Fix It - Modern War Institute (westpoint.edu) states that agencies "routinely faced challenges in maximizing the value of open-source information. More specifically, we encountered problems in three areas: collection, vetting and analysis, and sharing content. "
Collection: sheer amount of data and another related to language. loading social media and related software on an official computer is rarely an option.
Vetting and Analysis: how to analyze it to determine authenticity- Organizations and commands need access to an effective mechanism for analyzing a piece of content (tweet, text, video, image, etc.), including for use in messaging activities.
Sharing: no readily available means of storing it and sharing it with DoD, interagency, and international partners.
Based upon our research Marion Square believes the OSINT market should be a focus for technology vendors that can assist agencies in these areas especially companies that can provide novel AL/M solutions that are able to address both collection and vetting/analysis.
Government agencies that are particularly interested in OSINT for FY 24 include: the IC community, DoD Intel, DHS, Dept of State as well as State and Local Law enforcement agencies.
If you are interested in learning more about the OSINT market for the US Government, on March 26th, Marion Square will be hosting a webinar with our partner Carahsoft overviewing the Open Source Intelligence Market (OSINT) in Government. Register Now: Microsoft Virtual Events Powered by Teams