Recognition of activities in a video is one of the most fundamental tasks for understanding the scene. Our earlier work in this area was focused on context-aware activity recognition where the relationships between different objects and activities in the scene were exploited for recognition. More recent work has focused on predicting activities that may happen in the near future based on past observations. Here too, our work has focused on how to take advantage of contextual relationships within a video in order to predict the activities in the near-future. We also provided a natural language description of the activity. In fact, we presented one of the earliest works on near-future context-aware activity prediction in ICCV 2017 and ACCV 2014. In another recent work in this domain, we have leveraged concepts from human psychology to develop a novel paradigm for action anticipation.

Related to activity recognition, we have also looked into the problem of detecting anomalous activities in video. Anomalies are rare and need to be identified by understanding patterns of normalcy and detecting deviations from them. Our work was one of the first deep learning-based approaches for the detection of anomalous activities in video. 

This work has been supported by NSF, ONR, DARPA, NGA, and Google.