Search the Deep Web

Search engines such as Google index well over a trillion pages on the World Wide Web[1], but there is information on the web that common search engines don't reach. Most of this is in databases of information that need to be searched directly from the specific website. More famously (or infamously), a small pocket of the deep web is filled with hyper-secret communities who flock there to escape identification from authorities.

Steps

Deep Web Databases

  1. Locate databases with an ordinary search engine. You can use an ordinary search engine to find a "wildlife database," "hip hop database," or a similar term. Because the information in these databases can only be accessed by typing in a search term, not following a link, most search engine bots cannot find them, making that information part of the "deep web." The search engine can still lead you to the front page of the website, however, where you can use the search bar to make a more specific query.
  2. Use a more specialized search for databases. The Internet Public Library is no longer up and running. You can still use it but it's not being updated. You can visit searchengineguide.com to locate a specialized search engine for finding databases and information on a specific topic only.
  3. Research using the computer at an academic library. Libraries, especially ones attached to colleges and universities, often subscribe to a large number of pay-to-use databases, which contain information not found on ordinary search engines. Ask a librarian which databases they have available. You may even be able to access these databases using your library card information, but this depends on the specific library and database owner.
  4. Browse the Internet Archive. The Internet Archive project attempts to collect digital information to preserve long-term. Browse its many collections to find archives of websites that have since vanished, difficult to find video and audio clips, and even online copies of early video game systems.

Tor Network

  1. Understand the Tor network. This area of the deep web, sometimes called the Dark Net, is used for trades, conversations, and information that users want to keep private.[2] Visitors must use software called Tor to access this region of the web, visiting websites with the ".onion" domain. While much of this activity is illegal or legally grey, this is also an area used by journalists to talk to anonymous sources, and by people interested in the concept of an ultra-private internet.
    • Accessing this area of the deep web is legal, although that may not apply to activities you participate in there.
  2. Download the Tor Browser. Tor is a free service that lets you connect to web pages anonymously, making it extremely difficult for anyone to track your internet activity if you follow the right precautions. Many deep web communities can only be accessed through the Tor network, since they are founded on anonymity, privacy, and secrecy. Download the Tor Browser here to begin accessing this network.
    • Web pages on the Tor network tend to be unreliable, often going down for hours, days, or permanently. They can be slow to load as well, since Tor is routing your connection through other people's computers to protect your anonymity.
    • While Tor browsers exist for Android and iOS, these are not secure and not recommended.[3] Similarly, Tor add-ons for other browsers are not secure and are usually not supported by the Tor organization.
  3. Protect your anonymity. Accessing the deep web tor network is legal, but many people take advantage of the anonymity to engage in illegal activity. Taking these precautions is highly recommended to avoid malicious attacks or tracking by law enforcement agencies:
    • Click the "S" logo to the left of the Tor Browser address bar, and click "Forbid scripts globally."[4]
    • Turn on your Turn-on-Windows-Firewall or Turn-on-Firewall-for-Mac.
    • Never download any file from a Tor web page, not even a .pdf or .doc file. Torrent-sharing is especially insecure.
  4. Start with a deep web introduction from the "surface web" communities for up-to-date instructions and advice. Try posting on the subreddits /r/deepweb, /r/onions, or /r/Tor.
    • Most of the deep web links in this section are only accessible through Tor, not ordinary browsers.
  5. Use a deep web search engine. The deep web is intentionally difficult to explore and maintain, so these search engines may not be as effective as you are used to on the ordinary internet. In order to find a variety of results, try using several of them for each search, such as Torch (http://xmh57jrzrnw6insl.onion/) and TorSearch (http://kbhpodhnfxl3clb4.onion/).
    • If you're looking for a well-known deep web site, even a mainstream search engine like google can often find a link.[5]
  6. Use specialized deep web services. While the deep web is infamous for illegal activity, there are legitimate sites as well. Some are based on familiar concepts, like image sharing (e.g., http://www.zw3crggtadila2sg.onion/imageboard/). Others are more unique to deep web culture, such as secure whistleblowing sites (e.g., http://5r4bjnjug3apqdii.onion/) and eBook collections focused on subversive works (e.g., https://xfmro77i3lixucja.onion.lt/).
  7. Talk to deep web denizens. This area of the deep web loses major websites constantly, partly due to crackdowns on illegal activity, and partly because many of the websites are run by individuals or small teams without any funding. To find out the latest replacements or hot spots, talk to people using OnionChat (http://www.chatrapi7fkbzczr.onion/).

Tips

  • Vertical search engines also exist for specific types of content, for example http://www.findthatfile.com is a 'file specific' engine for non webpages, binary files from all types of sources.

Related Articles

Sources and Citations

  1. ↑ http://googleblog.blogspot.com/2008/07/we-knew-web-was-big.html
  2. ↑ http://nymag.com/daily/intelligencer/2014/11/explainer-why-the-fbi-shut-down-dark-net-sites.html
  3. ↑ http://www.fastcolabs.com/3026989/an-up-to-date-laymans-guide-to-accessing-the-deep-web
  4. ↑ https://www.torproject.org/docs/faq.html.en#DisableJS
  5. ↑ http://www.dailydot.com/technology/how-to-search-the-deep-web/
Retrieved from "https://kipkis.com/index.php?title=Search_the_Deep_Web&oldid=289935"