Ultrafast vision perception by neuromorphic optical flow

Saved in:

Bibliographic Details
Published in:	arXiv.org (Sep 10, 2024), p. n/a
Main Author:	Wang, Shengbo
Other Authors:	Gao, Shuo, Pu, Tongming, Zhao, Liangbing, Arokia Nathan
Published:	Cornell University Library, arXiv.org
Subjects:	Time domain analysis Visual tasks Three dimensional flow Two dimensional flow Optical data processing Data processing Algorithms Visual perception Optical flow (image analysis) Three dimensional motion Visual perception driven algorithms Two dimensional analysis
Online Access:	Citation/Abstract Full text outside of ProQuest
Tags:	Add Tag No Tags, Be the first to tag this record!

Description
Abstract:	Optical flow is crucial for robotic visual perception, yet current methods primarily operate in a 2D format, capturing movement velocities only in horizontal and vertical dimensions. This limitation results in incomplete motion cues, such as missing regions of interest or detailed motion analysis of different regions, leading to delays in processing high-volume visual data in real-world settings. Here, we report a 3D neuromorphic optical flow method that leverages the time-domain processing capability of memristors to embed external motion features directly into hardware, thereby completing motion cues and dramatically accelerating the computation of movement velocities and subsequent task-specific algorithms. In our demonstration, this approach reduces visual data processing time by an average of 0.3 seconds while maintaining or improving the accuracy of motion prediction, object tracking, and object segmentation. Interframe visual processing is achieved for the first time in UAV scenarios. Furthermore, the neuromorphic optical flow algorithm's flexibility allows seamless integration with existing algorithms, ensuring broad applicability. These advancements open unprecedented avenues for robotic perception, without the trade-off between accuracy and efficiency.
ISSN:	2331-8422
Source:	Engineering Database