Please use this identifier to cite or link to this item:
Scopus Web of Science® Altmetric
Type: Journal article
Title: Temporal pyramid pooling based convolutional neural network for action recognition
Author: Wang, P.
Cao, Y.
Shen, C.
Liu, L.
Shen, H.T.
Citation: IEEE Transactions on Circuits and Systems for Video Technology, 2016; 27(99):1-8
Publisher: Institute of Electrical and Electronics Engineers
Issue Date: 2016
ISSN: 1051-8215
Statement of
Peng Wang, Yuanzhouhan Cao, Chunhua Shen, Lingqiao Liu, Heng Tao Shen
Abstract: Encouraged by the success of Convolutional Neural Networks (CNNs) in image classification, recently much effort is spent on applying CNNs to video based action recognition problems. One challenge is that video contains a varying number of frames which is incompatible to the standard input format of CNNs. Existing methods handle this issue either by directly sampling a fixed number of frames or bypassing this issue by introducing a 3D convolutional layer which conducts convolution in spatial-temporal domain. In this paper we propose a novel network structure which allows an arbitrary number of frames as the network input. The key of our solution is to introduce a module consisting of an encoding layer and a temporal pyramid pooling layer. The encoding layer maps the activation from previous layers to a feature vector suitable for pooling while the temporal pyramid pooling layer converts multiple frame-level activations into a fixed-length video-level representation. In addition, we adopt a feature concatenation layer which combines appearance information and motion information. Compared with the frame sampling strategy, our method avoids the risk of missing any important frames. Compared with the 3D convolutional method which requires a huge video dataset for network training, our model can be learned on a small target dataset because we can leverage the off-the-shelf image-level CNN for model parameter initialization. Experiments on three challenging datasets, Hollywood2, HMDB51 and UCF101 demonstrate the effectiveness of the proposed network.
Keywords: Temporal pyramid pooling; action recognition; convolutional neural network
Rights: © IEEE
DOI: 10.1109/TCSVT.2016.2576761
Appears in Collections:Aurora harvest 8
Electrical and Electronic Engineering publications

Files in This Item:
File Description SizeFormat 
  Restricted Access
Restricted Access6.02 MBAdobe PDFView/Open

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.