The goal in this application is to recognise various types of interesting human activity, such as when two people meeting or moving together. We have been developing complex event recognition technology for activity recognition that operates on a stream of ‘short-term activities’, detected on individual video frames by visual information processing techniques.
We have evaluated our technology on the benchmark CAVIAR dataset. Below, are a few videos from this dataset illustrating our complex event recognition technology. In each video, the ground truth label, provided by the CAVIAR team, is displayed at the top of the bounding box, while the recognised activity is displayed at the bottom. When our recognition agrees with the ground truth, both ground truth and recognition labels are green. When there is no ground truth and there is recognition, the bottom label is red (and there is no top label). When there is ground truth and there is no recognition, the top label is red (and there is not bottom label). Finally, when our recognition disagrees with the ground truth, the top label is green and the bottom is red.
Run-Time Event Calculus (RTEC)
RTEC is an open-source, logic programming implementation of the Event Calculus, designed to compute continuous temporal projection queries for pattern matching on data streams. The videos below display event recognition by means of RTEC, when using manually constructed patterns for ‘fighting’ (two people are fighting).
Artikis A., Sergot M. and Paliouras G. An Event Calculus for Event Recognition. IEEE Transactions on Knowledge and Data Engineering (TKDE), 27(4): 895-908, 2015. (pdf)
Artikis A., Sergot M. and Paliouras G. A Logic Programming Approach to Activity Recognition, 2nd ACM International Workshop on Events in Multimedia, 3-8, 2010. (pdf)
Inductive Logic Programming-based Online Learning
OLED is an online learner constructing logic programs expressing Event Calculus patterns, that may be subsequently used for event recognition. OLED extends Inductive Logic Programming techniques for highly efficient learning over data streams. The videos below display event recognition when using the patterns for ‘meeting’ (two people are meeting) and ‘moving’ (two people are moving together), as constructed by OLED.
Katzouris N., Artikis A., Paliouras G.: Online Learning of Event Definitions, Theory and Practice of Logic Programming (TPLP), special issue for the 32nd International Conference of Logic Programming (ICLP 2016). (pdf)
Katzouris N, Michelioudakis E., Artikis A. and Paliouras G. Online Learning of Weighted Relational Rules for Complex Event Recognition, European Conference on Machine Learning (ECML-PKDD) 2018. (pdf)
Katzouris N., Artikis A. and Paliouras G., Parallel Online Event Calculus Learning for Complex Event Recognition, Future Generation Computer Systems, Vol. 94, pp. 468-478, 2019. (pdf)
Markov Logic-based Online Learning
OSLα is an online learner constructing Event Calculus patterns in Markov Logic, that may be subsequently used for (probabilistic) event recognition. The videos below illustrate the use of OSLα, by displaying event recognition when using the patterns for ‘meeting’ (two people are meeting) and ‘moving’ (two people are moving together), as constructed by OSLα.
Micheloudakis V., Skarlatidis A., Paliouras G and Artikis A: OSLa: Online Structure Learning using Background Knowledge Axiomatization, European Conference onMachine Learning (ECML-PKDD) 2016 (pdf)
Michelioudakis E., Artikis A. and Paliouras G., SemiSupervised Online Structure Learning for Composite Event Recognition, Machine Learning, 108(7), pp. 1085-1110, 2019. (pdf)