Please use this identifier to cite or link to this item:
Scopus Web of Science® Altmetric
Type: Journal article
Title: Structured learning of human interactions in TV shows
Author: Patron-Perez, A.
Marszalek, M.
Reid, I.
Zisserman, A.
Citation: IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012; 34(12):2441-2453
Publisher: IEEE
Issue Date: 2012
ISSN: 0162-8828
Statement of
Alonso Patron-Perez, Marcin Marszalek, Ian Reid, and Andrew Zisserman
Abstract: The objective of this work is recognition and spatiotemporal localization of two-person interactions in video. Our approach is person-centric. As a first stage we track all upper bodies and heads in a video using a tracking-by-detection approach that combines detections with KLT tracking and clique partitioning, together with occlusion detection, to yield robust person tracks. We develop local descriptors of activity based on the head orientation (estimated using a set of pose-specific classifiers) and the local spatiotemporal region around them, together with global descriptors that encode the relative positions of people as a function of interaction type. Learning and inference on the model uses a structured output SVM which combines the local and global descriptors in a principled manner. Inference using the model yields information about which pairs of people are interacting, their interaction class, and their head orientation (which is also treated as a variable, enabling mistakes in the classifier to be corrected using global context). We show that inference can be carried out with polynomial complexity in the number of people, and describe an efficient algorithm for this. The method is evaluated on a new dataset comprising 300 video clips acquired from 23 different TV shows and on the benchmark UT--Interaction dataset.
Keywords: Human interaction recognition; video retrieval; structured SVM
Rights: © 2012 IEEE
DOI: 10.1109/TPAMI.2012.24
Appears in Collections:Aurora harvest 7
Computer Science publications

Files in This Item:
File Description SizeFormat 
  Restricted Access
Restricted Access1.43 MBAdobe PDFView/Open

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.