Neuromorphic vision is a branch of Computer Vision which operates using bio-inspired sensors distinct from traditional cameras. This type of sensors record only changed pixel intensities at microseconds precision rate. The high temporal resolution enables capturing events that appear within the range of million time steps in a second, thus acquiring very detailed information, which is not accessible via frame based RGB sensors. The event, in this context, is a record about changed pixel in the form of four-dimensional tuple (𝑥,𝑦,𝑝,𝑡) that documents an intensity change at the position (𝑥,𝑦), the time t and the polarity 𝑝∈(0,1).
ISSAI researchers Ulzhan Bissarinova, Tomiris Rakhimzhanova, Daulet Kenzhebalin, and founding director Atakan Varol have published the paper featuring first large and varied event face dataset called Faces in Event Streams (FES), accompanied by 12 models that detect faces and facial landmarks within event streams with an mAP50 score higher than 90%. The dataset contains 689 minutes of recorded event streams, as well as 1.6 million annotated faces. Researchers also compare the performance of a frame based model and event based models, as well as accumulation techniques for event models based on fixed number of events and fixed time. The experiments conclude that event based models outperform the frame based model, and time based accumulation techniques are a better fit for detecting faces on the FES dataset.
The research results might be further extended as a base face detector for face recognition and emotion recognition from event streams tasks, as well as for intensity reconstruction. The FES dataset and pre-trained models are publicly available for further research at GitHub.