Convolutional Approach to learning Temporal Feature Effectively in Transformer

초록

In the video classification task, a well-performing deep learning model is likely to extract proper temporal features to classify the data. However, we found out several problems of attention-based TimeSformer[2] related to extracting temporal features, and replaced the time attention module in the TimeSformer with the 3D convolution module for better temporal feature processing. Through several experiments and visualization results, we demonstrate that the 3D convolution module can extract more accurate temporal features of video data than the time-attention module.