OSINT Radar
Web & URL OSINT Operational

Fast Osint Crawler

github.com

Photon is a fast OSINT crawler extracting URLs, emails, files, subdomains, and metadata from any target website for investigators.

Free Open Source #Fast Osint Crawler #GitHub OSINT tools #GitHub OSINT resources #crawler #fast #osint #capabilities #code #designed #developer
Open tool

Pivot map

You have url domain image file
You get page content infrastructure mentions metadata timestamps

Use this map to decide whether Fast Osint Crawler accepts the lead you are holding, and what kind of lead it may return for the next step.

Pivot from here

Outputs from Fast Osint Crawler can become inputs for the next tool. These are the most relevant follow-on pivots in the library.

Investigator Use

Photon is an open-source Python-based fast OSINT web crawler available on GitHub that extracts intelligence from websites automatically. It harvests URLs, subdomains, emails, JavaScript files, social media links, and other data points from target websites and their linked pages.


For OSINT investigators, Photon automates the intelligence extraction phase of web investigation — rather than manually reviewing page source code, following links, and recording extracted data, Photon systematically crawls a target site and extracts structured intelligence automatically.


Email harvesting from web content is one of Photon's primary outputs. When directed at a target organization's website, it extracts all email addresses visible in page content — building the email inventory needed for subsequent OSINT research.


URL and subdomain discovery through systematic link following reveals the complete structure of a target website including internal pages, external links, and subdomains that may not be obvious from the homepage.


JavaScript file analysis from Photon's extraction can reveal API endpoints, hardcoded credentials, internal service names, and other sensitive technical information that developers have inadvertently included in client-side JavaScript.


Social media profile links extracted from target websites provide direct leads to the organization's or individual's social media presence across platforms.


Custom key extraction allows investigators to configure Photon to search for specific patterns — custom regex for finding specific data types relevant to a particular investigation.


Photon's speed-optimized crawler handles large websites efficiently, with configurable depth limits, thread counts, and extraction filters that allow investigators to balance thoroughness with time constraints.


Operational considerations: Photon makes direct HTTP requests to target websites — use in investigation environments with appropriate anonymization if source IP concealment is required. Only crawl authorized targets.


Document target websites crawled, crawl depth, and all extracted data categories with timestamps.

Before You Pivot

Record Context

Capture the target, search terms, and why this source is relevant before you leave the page.

Preserve Evidence

Archive volatile pages, save screenshots, and keep timestamps for anything that may change.

Corroborate

Treat one tool as a lead source. Confirm important findings with independent sources.

Related tools