OSINT Radar
Dark Web OSINT Operational

Tor Crawl

github.com

Crawl and extract (regular or onion) webpages through TOR network

Free Open Source #Tor Crawl #Dark Web OSINT tools #Dark Web OSINT resources #crawl #tor #onion #capabilities #dark #darknet #discovery
Open tool

Pivot map

You have domain url
You get onion services mentions

Use this map to decide whether Tor Crawl accepts the lead you are holding, and what kind of lead it may return for the next step.

Pivot from here

Outputs from Tor Crawl can become inputs for the next tool. These are the most relevant follow-on pivots in the library.

Investigator Use

TorCrawl.py is an open-source Python-based Tor network web crawler available on GitHub. Developed by MikeMeliz, it is designed to crawl websites through the Tor anonymity network, extracting links, page content, and metadata from both .onion hidden services and clearnet sites accessed via Tor exit nodes.


For OSINT investigators, TorCrawl.py provides automated capability for mapping the structure and content of dark web sites and for accessing clearnet sites through Tor when operational anonymity during investigation is required. The tool's ability to traverse the link graph of a target site systematically makes it useful for comprehensive dark web site analysis.


Primary investigative applications include: mapping the full link structure of a dark web marketplace or forum to understand its scope and architecture, extracting all external references from a target dark web site to identify affiliated infrastructure, archiving page content from dark web sites that may go offline before manual review is complete, and accessing clearnet investigation targets through Tor when operational anonymity is required.


TorCrawl.py provides configurable crawl depth and scope parameters, allowing investigators to limit crawling to specific subdirectories or to expand coverage to linked external sites. This control is important for keeping investigations focused and avoiding unnecessary exposure to out-of-scope content.


The tool outputs structured data including link lists, page content, and metadata, which can be processed with standard data analysis tools for further investigation. The extracted link network can be imported into graph visualization tools to map the relationship structure of a dark web ecosystem.


Technical requirements include Python 3, the Tor service running locally, and the stem library for Tor circuit management. The tool automatically routes all traffic through Tor using the configured SOCKS proxy.


Operational security, legal considerations, and authorization requirements identical to other dark web crawling tools apply. Always operate in an isolated environment, document all crawl parameters and outputs, and verify legal authority before crawling any target system.

Before You Pivot

Record Context

Capture the target, search terms, and why this source is relevant before you leave the page.

Preserve Evidence

Archive volatile pages, save screenshots, and keep timestamps for anything that may change.

Corroborate

Treat one tool as a lead source. Confirm important findings with independent sources.

Related tools