Popular TV shows such as Mr. Robot and movies have fueled the perception that the Dark Web is a massive network of criminal sites engaged in all manner of illegal activities. The reality is a bit more prosaic: it isn’t all that large, after all.
The Dark Web is shrouded in mystery, helped by the fact that it requires special tools—software and configurations—to access. Tor is one such tool. However, the phrase Dark Web has been overused to refer to any underground marketplace or criminal forum. The term is often conflated with the marketplaces on the Deep Web, which refers to parts of the Web not indexed by search engines, often because they are behind paywalls or some other type of login mechanism.
Research from threat intelligence company Recorded Future found the number of live, accessible .onion sites amount to less than 0.005 percent of surface web domains. To put in context, there are about 200 million surface Web domains. The research focused on estimating the full size of a reachable Tor network by counting .onion sites.
“The popular iceberg metaphor that describes the relationship of the surface web and dark web is upside down,” wrote Recorded Future data scientists Garth Griffin and Juan Sanchez, as unlike an iceberg, the part of the web that we can see is much larger than the web we don’t see.
Recorded Future researchers used inbound links to map the .onion sites on the Dark Web. On the surface Web—the Web that is familiar to most people—inbound links to a site help determine the site’s popularity. More inbound links mean more ways for people to reach that site. The same goes with the Dark Web—researchers began with a set of onion sites pulled from public lists and from its internal research. They crawled 260,000 pages and found 55,828 different onion domains, but only 8,416 were observed to be live on the Tor network. Only 15 percent were live sites.
Criminals Go Anywhere
Whenever there is any discussion about online criminal activity, or some kind of illegal activity in a marketplace, the immediate conclusion is that it must be happening on the Dark Web. There are places on Dark Web for illicit activity—they form “a tiny portion of onion sites, a set of invitation-only and generally unpublicized communities buried in the most shadowy corners of the internet,” the researchers said.
The unpleasant truth is that criminals get together wherever they can, and wherever makes sense for them, said Chris Camacho, the chief strategy officer of threat intelligence company Flashpoint. Stolen data can turn up in all kinds of places. Some are Tor sites, some are buried away in log-in only sites on the Deep Way, but many others are traded and sold in the open right on surface web. Some of the activity don’t even happen on the Web, as it may happen on IRC channels, chat apps such as WhatsApp, or sites such as Discord.
The bulk of the criminal activity online happens on chat apps, followed by password-protected forums (not on Dark Web), Camacho said.
Recorded Future noted the same, that “much criminal activity happens on sites not requiring any special protocols to access, such as public social media sites like Twitter or messaging services like WhatsApp and Telegram.”
Understanding that the Dark Web is smaller than expected would help manage expectations from the enterprise defender’s side. Just monitoring the Dark Web will uncover only some bad things—perhaps there will be an early alert exposed payment card numbers—but if the forum selling access to compromised servers is not on the Dark Web, that laser-focus on monitoring the Dark Web is not helpful.
Enterprises “need a partner to help make sure the coverage is comprehensive and that they are seeing everything,” Camacho said.
Criminals that are intent on making money are going to do keep moving in order to stay hidden: they will use anonymizing tools and services, and they will pivot to new methods and sites if they are more effective. If one marketplace shuts down, they go to another. If a chat group gets too noisy, they branch off into smaller conversations.
Defenders have the complex task of trying to find all the places they can hide to find out what they are up to without exposing themselves, Camacho said.