DARPA is usually so tight-lipped about the technologies it seeks that we can do little but read through bidding solicitations and speculate on what the agency is up to. But in a new request, the DoD's blue-sky research team is asking for help shining light on the deepest corners of the Pentagon's archives, asking industry and academia for help in developing technology that will help the government sort through its endless pool of stored information for material suitable for declassification.
In free democracies (in theory, anyhow), the government is only supposed to withhold information deemed sensitive enough to be a threat to the nation's security or overall well-being. But bureaucracies have a tendency to grow and accrue, leaving huge offices of the state with reams upon reams of information – so much that they have no clue what a lot of it is.
DARPA is seeking a technological solution to this problem of policy by developing some means of sorting through the mess automatically to find information that is ready to see the light of day. From DARPA's solicitation: "Improving the capability of departments and agencies to identify still-sensitive information and to make declassified information available to the public are integral parts of the classification system."
How do you do that? Well, semantically it's possible to comb through data – at least the stuff that's been digitized to some degree – searching for keywords that suggest a document is ready for public view. But to fully automate the process, you need something more: a subjective review of the topic to ensure that through the right (or wrong) combination of keywords or phrasing that something labeled "For Your Eyes Only" doesn't turn up in Google searches.
That sort of value weighing and subjective decision-making requires something more, and that seems to be what DARPA is angling for. There's no telling what the DoD's eventual solution might look like, but we'll keep you posted – that is, if the Pentagon makes the information publicly available.
I am willing to bet $1 million (on credit) that Google already has a solution for this. Any takers?
If this does go through, it will be a silent renaissance in the private sector as there are buildings full of useful reports and technical papers that will lead to myriad innovations.
I can see a scandal in the coming years of a report that was not supposed to be released that got "accidentaly decalrifed releasable by the system"
Wiki leaks will take care of it all eventually so sit back and stuff some popcorn and drink a beer the show is about to start.