Download PDFOpen PDF in browser

Action Prediction During Human-Object Interaction Based on Dtw and Early Fusion of Human and Object Representations

EasyChair Preprint no. 6218

10 pagesDate: August 1, 2021


Action prediction is defined as the inference of an action label while the action is still ongoing. Such a capability is extremely useful for early response and further action planning. In this paper, we consider the problem of action prediction in scenarios involving humans interacting with objects. We formulate an approach that builds time series representations of the performance of the humans and the objects. Such a representation of an ongoing action is then compared to prototype actions. This is achieved by a Dynamic Time Warping (DTW)-based time series alignment framework which identifies the best match between the ongoing action and the prototype ones. Our approach is evaluated quantitatively on three standard benchmark datasets. Our experimental results reveal the importance of the fusion of human- and object-centered action representations in the accuracy of action prediction. Moreover, we demonstrate that the proposed approach achieves significantly higher action prediction accuracy compared to competitive methods.

Keyphrases: action classification, action label, action prediction, action prediction accuracy, computer vision, distance matrix, Dynamic Time Warping (DTW), Early Action Classification, Global Alignment Kernel (GAK)., human-object interaction, msr daily activity dataset, Observation ratio, short term action prediction, Soft Dynamic Time Warping (Soft DTW), Time series alignment, Trimmed video

BibTeX entry
BibTeX does not have the right entry for preprints. This is a hack for producing the correct reference:
  author = {Victoria Manousaki and Konstantinos Papoutsakis and Antonis Argyros},
  title = {Action Prediction During Human-Object Interaction Based on Dtw and Early Fusion of Human and Object Representations},
  howpublished = {EasyChair Preprint no. 6218},

  year = {EasyChair, 2021}}
Download PDFOpen PDF in browser