Methods and apparatus for optical flow estimation with contrastive learning

Apr 1, 2024·
Zhiqi Zhang
,
Pan Ji
,
Nitin Bansal
,
Changjiang Cai
,
Qingan Yan
,
Xiangyu Xu
,
Huangying Zhan
,
Yi Xu
· 0 min read
Abstract
A method for a computing system includes determining a first and second features map in response to a first and second images, implementing a gated recurrent unit (GRU) to determine a pixel-wise flow prediction in response to the first features map and the second features map, determining a warped feature map in response to the second image map, implementing a feature-wise contrastive loss function to determine a feature-wise loss in response to first features in the first image map and second features in the warped features map, determining a pixel-wise flow loss in response to the pixel-wise flow prediction and to pixel-wise ground truth data, and modifying parameters of the GRU in response to the pixelwise flow loss and to the feature-wise loss.
Type