Lately, the Office of the Director of National Intelligence (ODNI) revealed a fresh approach for open-source intelligence (OSINT) and labeled OSINT as the “INT of first resort”. Public and private sector establishments are acknowledging the significance that the discipline can present but are also realizing that the immense expansion of digital data in recent years has inundated many traditional OSINT techniques. Fortunately, Artificial Intelligence (AI) and Machine Learning (ML) are commencing to have a revolutionary effect on the future of information collection and analysis.
What Constitutes Open-Source Intelligence (OSINT)?
Open-Source Intelligence pertains to the compilation and scrutiny of information from publicly accessible outlets. These outlets can encompass conventional media, social media platforms, academic publications, government reports, and any other data that is freely obtainable. The primary hallmark of OSINT is that it does not involve secretive or undercover methods of information collection such as human intelligence or social engineering. If I could have obtained data during my tenure working for the U.S. Government but I am no longer able to do so as a civilian, that does not qualify as OSINT.
Traditionally, OSINT has been a labor-intensive procedure involving several pivotal stages:
- Source Identification: Analysts ascertain which public sources are probable to contain pertinent information.
- Data Compilation: Information is amassed from these sources, frequently through manual searches or web scraping tools.
- Data Arrangement: The gathered information is systematized and structured for analysis.
- Analysis: Proficient analysts scrutinize the data to discern patterns, trends, and insights.
- Compilation: Discoveries are amalgamated into reports for decision-makers to facilitate more informed decisions.
While potent, this strategy encounters restrictions due to the sheer abundance of available information. Human analysts grapple with processing everything manually and valuable insights might be concealed in intricate patterns that are strenuous for humans to detect. This is where AI/ML can furnish a substantial advantage in how information can be assembled, processed, and analyzed, thereby releasing the human analyst to concentrate on things they are distinctly qualified for such as offering context. As an additional advantage, this transition frequently boosts morale as humans expend less time on dreary processing tasks and more time analyzing and evaluating information.
Activities where AI/ML can offer prompt advantage encompass:
- Managing Enormous Data Quantities: AI systems can scrutinize and evaluate colossal volumes of data at speeds surpassing human capabilities. This enables OSINT practitioners to cast a noticeably broader net than previously feasible and still manage the outcomes.
- Real-time Evaluation: The cascade of information flow in today’s digital realm is staggering. AI-driven OSINT tools can supervise and evaluate data streams in real-time, furnishing up-to-the-minute intelligence and enabling swift responses to emergent scenarios.
- Multi-language and Multimodal Evaluation: AI can dismantle language barriers by translating and appraising content in multiple languages concurrently. Moreover, it can process diverse data formats – text, images, audio, and video – in an amalgamated approach, offering a more inclusive intelligence picture. Numerous of these capabilities such as OpenAI’s Whisper can be employed offline, thereby eliminating any concerns regarding operational security (OPSEC).
- Prognostic Analytics: By evaluating historical data and contemporary trends, AI can aid in predicting future events or behaviors, contributing a proactive dimension to OSINT.
- Streamlining Routine Activities: AI can aid in automating numerous time-consuming facets of OSINT, such as data collection and initial filtering, freeing human analysts to focus on higher-level analysis and decision-making. Tasks that were previously extremely challenging if not impossible to execute, such as precise sentiment analysis, are now effortless.
At SANS Network Security the SEC497 Practical OSINT course and the SEC587 Advanced OSINT course will furnish students with hands-on experience deploying these AI capabilities to not only enhance productivity but also unveil novel possibilities.
Although no technology is flawless, and we must mull over the plausible consequences that a hallucination could trigger before we implement AI, crucial technologies currently being employed for OSINT encompass:
- Natural Language Processing (NLP): NLP permits machines to comprehend, interpret, and produce human language. In OSINT, NLP is critical for:
- Sentiment analysis of social media posts
- Entity recognition to identify people, organizations, and locations in text
- Topic modeling to categorize extensive volumes of text data
- Machine translation for multilingual intelligence gathering
- Computer Vision: This technology empowers machines to interpret and analyze visual information. In OSINT, computer vision is utilized for:
- Facial recognition in images and videos
- Facial comparisons to identify if the same person is depicted in multiple images
- Object detection in imagery
- Optical character recognition (OCR) to extract text from images
- Scene comprehension in video footage
- Machine Learning and Data Mining: How frequently have you heard “those who don’t know history are doomed to repeat it”? Machine Learning epitomizes that notion as it enables systems to learn from data and enhance their performance over time. In OSINT, they are utilized for:
- Predictive analytics to predict trends or events
- Anomaly detection to identify unusual patterns or behaviors
- Grouping and categorization of data for easier analysis
- Network analysis to comprehend relationships between entities
I’ve been engaged in OSINT for nearly two decades and this is unequivocally the most vibrant and thrilling period I’ve observed with fresh advancements in the domain transpiring literally daily. If you’re going to be at Network Security in Las Vegas this September, I anticipate discussing how this capability can enhance our efficiency and effectiveness today, as well as what we can anticipate in the future.
Haven’t registered for SANS Network Security yet? Peruse this page to discover all that awaits!
Note: This article is proficiently composed by Matt Edmondson, a SANS Principal Instructor and Principal at Argelius Labs, with a decade of professional OSINT experience.
About Author
