Attention Filtering for Multi-person Spatiotemporal Action Detection on\n Deep Two-Stream CNN Architectures

Résumé fourni par la source

Action detection and recognition tasks have been the target of much focus in\nthe computer vision community due to their many applications, namely, security,\nrobotics and recommendation systems. Recently, datasets like AVA, provide\nmulti-person, multi-label, spatiotemporal action detection and recognition\nchallenges. Being unable to discern which portions of the input to use for\nclassification is a limitation of two-stream CNN approaches, once the vision\ntask involves several people with several labels. We address this limitation\nand improve the state-of-the-art performance of two-stream CNNs. In this paper\nwe present four contributions: our fovea attention filtering that highlights\ntargets for classification without discarding background; a generalized binary\nloss function designed for the AVA dataset; miniAVA, a partition of AVA that\nmaintains temporal continuity and class distribution with only one tenth of the\ndataset size; and ablation studies on alternative attention filters. Our\nmethod, using fovea attention filtering and our generalized binary loss,\nachieves a relative video mAP improvement of 20% over the two-stream baseline\nin AVA, and is competitive with the state-of-the-art in the UCF101-24. We also\nshow a relative video mAP improvement of 12.6% when using our generalized\nbinary loss over the standard sum-of-sigmoids.\n

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.