DARPA Links Human Brainwaves, Improved Sensors and Cognitive Algorithms to Detect Threats

DARPA's CT120 CameraDARPA is taking "computer vision" to a whole new level. The group is combining new-wave sensors and  cognitive algorithms with brain scans to enable soldiers to process an unbelievable amount of information.

In 2008, the group launched the Cognitive Technology Threat Warning System (CT2WS) program with the goal of maximizing warfighters’ awareness of their surroundings by developing portable visual threat detection devices. CT2WS created technology that can identify up to 91 percent of targets during testing with extremely low false-alarm rates and can widen a soldier's field of view to 120 degrees when all components of the kit are used in tandem. By incorporating a commercial radar (the Cerberus Scout surveillance system), target detection reached 100 percent.

“DARPA set out to solve a common challenge for forward troops: how can you reliably detect potential threats and targets of interest without making it a resource drain?” said Gill Pratt, DARPA program manager. “The prototype system has demonstrated an extremely low false alarm rate, a detection rate in the low 90s, all while reducing the load on the operator.”

The CT2WS system includes a 120-megapixel, tripod-mounted, electro-optical video camera with a 120-degree field of view; cognitive visual processing algorithms that can be run on laptops to identify potential targets and cue images for operator review; and an EEG cap that monitors the operator’s brain signals and records when the operator detects a threat. Pretty cool, right?

As DARPA explains it, CT2WS built on the concept that humans are inherently adept at detecting the unusual. Even though a person may not be consciously aware of movement or of unexpected appearance, the brain detects it and triggers a brain signal that is thought to be involved in stimulus evaluation or categorization. By improving the sensors that capture imagery and filtering results, a user wearing an EEG cap can rapidly view the filtered image set and let the brain’s natural threat-detection ability work. Amazingly, users are shown approximately 10 images per second, on average. Despite that quick sequence, brain signals indicate to the computer which images were significant.

The use of EEG-based human filtering significantly reduces the amount of false alarms. The cognitive algorithms can also highlight events that would otherwise be considered irrelevant but are actually indications of threats or targets, such as a bird flying by or a branch’s swaying. In testing of the full CT2WS kit, absent radar, the sensor and cognitive algorithms returned 810 false alarms per hour. When a human wearing the EEG cap was introduced, the number of false alarms dropped to only five per hour, out of a total of 2,304 target events per hour, and a 91 percent successful target recognition rate.

Field tests of the CT2WS system were conducted in desert terrain at Yuma Proving Ground in Arizona, in tropical terrain in Hawaii, and in open terrain at California’s Camp Roberts. DARPA provided a final demonstration of the CT2WS system to Army officials at Fort Belvoir, Virginia.