The US Defense Advanced Research Projects Agency (DARPA) has selected three teams of researchers led by Raytheon, BAE Systems, and Northrop Grumman to develop event-based infrared (IR) camera technologies under the Fast Event-based Neuromorphic Camera and Electronics (FENCE) program. It is designed to make computer vision cameras more efficient by mimicking how the human brain processes information.
Today’s state-of-the-art cameras work well with scenes that have few changes to track, and the imagery is relatively simple. However, their capabilities fail in highly cluttered and dynamic scenes, limiting their use among many military applications.
DARPA’s FENCE program aims to develop a new class of low-latency, low-power, event-based infrared focal plane array (FPA) and digital signal processing (DSP) and machine learning (ML) algorithms. The development of these neuromorphic camera technologies will enable intelligent sensors that can handle more dynamic scenes and aid future military applications.
Event-based imaging sensors operate asynchronously and only transmit information about pixels that have changed. This means they produce significantly less data and operate with much lower latency and power.
“Neuromorphic refers to silicon circuits that mimic brain operation; they offer sparse output, low latency, and high energy efficiency,” said Dr. Whitney Mason, the program manager leading the FENCE program. “Event-based cameras operate under these same principles when dealing with sparse scenes, but currently lack advanced ‘intelligence’ to perform more difficult perception and control tasks.”
Researchers from Raytheon, BAE, and Northrop will work to develop an asynchronous read-out integrated circuit (ROIC) with low latency and a processing layer that integrates with ROICs to detect relevant ‘spatial and temporal signals.’ According to DARPA, together, the ROIC and processing layer will enable an integrated FENCE sensor to operate at less than 1.5W of power.
“The goal is to develop a ‘smart’ sensor that can intelligently reduce the amount of information that is transmitted from the camera, narrowing down the data for consideration to only the most relevant pixels,” noted Mason.
In February, DARPA awarded initial Phase I design contracts for its Longshot Unmanned Aerial Vehicle (UAV) program.
