DARPA wants to build a better search engine and fight crime
The defense research agency seeks to create a way to search the hidden, often illegal world of the 'deep Web.'
Wed, Feb 12, 2014 at 03:36 PM
Photo: John Platt
Is there a better way to search the Internet and root out crime in the process? The Defense Advanced Research Projects Agency (DARPA) thinks there must be. The agency — which funds the development of new military technologies such as robots, satellites and body armor — is looking for a partner organization to help the agency create a new kind of Web search engine. They are calling the project "Memex," a combination of the words "memory" and "index."
So what would make Memex different? The most important aspect is that DARPA is looking for ways to technologically index the "deep Web" of the Internet. That's the hidden stuff that most search engines such as Google and Bing don't scan: forums, chat rooms and other semi-hidden parts of the Web.
Memex could have a lot of uses, but DARPA has an immediate purpose in mind. Mining the "deep Web" could help ferret out human trafficking operations, which frequently operate in those little-monitored corners of the Internet. The goal of the system is to supersede sites that have enabled software that currently prevents them from being "crawled" by today's search engines. DARPA calls human trafficking "a growing industry of modern slavery" and says "an index curated for the counter trafficking domain, including labor and sex trafficking, along with configurable interfaces for search and analysis will enable a new opportunity for military, law enforcement, legal, and intelligence actions to be taken against trafficking enterprises."
Law enforcement and military operations could definitely use a system like this. Time magazine covered the "deep Web" last year, writing that it is an "electronic haven for thieves, child pornographers, human traffickers, forgers, assassins and peddlers of state secrets and loose nukes." Many of these sites are only accessible with specific software programs — not your normal Web browsers — something Memex could, in theory, also accomplish. Ironically enough, the deep Web was actually created by the U.S. government as a tool for espionage agents and law enforcement, but it has, over the past decade, become used by a wide range of people seeking either simple privacy or a secret way to conduct illicit business.
Memex is also about organizing all of this content — especially content that is spread across multiple Web pages — and creating new ways to search this information and provide related insight. DARPA also hopes that searches will form a collective, collaborative resource, so previous searches can be archived and compared against future results. This could have important implications for law enforcement.
DARPA opened the Memex project for proposals last week. Abstracts for proposals are due Feb. 25 and companies will have until April 8 to submit their final proposals. A finished system would be expected in a few years.
Related on MNN: