Frame-based vision systems generate frames in a continuous and a repetitive way, even if there is any new relevant information to transmit. The amount of data that can generate can be sometimes prohibitive. This made them inappropiate for certain applications with limited bandwidth and low power consumption, i. e. surveillance or visual inspection.
On the other hand, biological systems transmit the information in an efficient way, removing redundancy and optimizing the use of energy. The human retina performs a spatio-temporal filtering operation to remove redundancy and transmit only the relevant information to the brain.
In particular, spatial contrast detection provides a very compressed data flow. By detecting the spatial contrast, edges and shapes can be recognized. Starting from the previous design of Zaghloul and Boahen (Part_I, Part II), I designed and tested a bio-inspired AER (Address Event Representation) spatial contrast retina (PDF). This project was done with the invaluable collaboration of my former advisors Bernabe Linares and Teresa Serrano at the Institute of Microelectronics of Seville (IMSE).
We designed a bio-inspired pixel. In this pixel, transistors model the retina cells and their interactions. The sensor has a signed output and faced some of the traditional limitations of these kind of sensors: high mismatch, biasing dependence with the illumination conditions, and low controlability.
We also added some additional features to the sensor and a Time-to-First Spike operation mode. Traditionally, these kind of sensors have been plagued by mismatch. They operate with transistors in subthreshold inversion and very low bias currents (100pA). To face this problem, we designed a Calibration_System for large neuromorphic arrays. The system only needs to be calibrated once and the calibration procedure is not very sensitive to the illumination conditions.
We can see below some snapshots taken with the spatial contrast retina. Although the resolution of the sensor was low (32x32 pixels), we were able to take some snapshots of real world images. The sensor can detect the sign of the contrast. Grey level indicates no contrast. Colors ranging from grey to white represent positive contrast. Colors ranging from black to grey represent negative contrast.
With the Time-to-First Spike operation mode, the sensor is globally reset. After this, pixels with more relevant information will spike first. After receiving a certain number of events, it will be enough to perform recognition. Thus, this is also an efficient way to remove noise and redundancy and save energy. With this mode, the time between consecutive reset signals can be adjusted dynamically according to the level of information of the visual scene. We can see below some images taken with this operation mode after receiving different number of events. We can recognize the shape of the object and it is not necessary to continue receiving data after counting a certain number of events. At this point, the sensor can enter in standby operation mode and save energy.
In the video displayed below, we can see one ball pen that I was moving across the sensor visual field.
The following table summarizes the main sensor features:
Main Project Publications
Back
