The goal in this application is to recognise various types of interesting human activity, such as when two people meeting or moving together. We have been developing complex event recognition technology for activity recognition that operates on a stream of ‘short-term activities’, detected on individual video frames by visual information processing techniques.
We have evaluated our technology on the benchmark CAVIAR dataset. Below, are a few videos from this dataset illustrating our complex event recognition technology. In each video, the ground truth label, provided by the CAVIAR team, is displayed at the top of the bounding box, while the recognised activity is displayed at the bottom. When our recognition agrees with the ground truth, both ground truth and recognition labels are green. When there is no ground truth and there is recognition, the bottom label is red (and there is no top label). When there is ground truth and there is no recognition, the top label is red (and there is not bottom label). Finally, when our recognition disagrees with the ground truth, the top label is green and the bottom is red.
Reasoning using the Event Calculus
The videos below display event recognition by means of RTEC, an open-source logic programming implementation of the Event Calculus, when using manually constructed patterns for ‘fighting’ (two people are fighting).
Artikis A., Makris E. and Paliouras G., A Probabilistic Interval-based Event Calculus for Activity Recognition.
In Annals of Mathematics and Artificial Intelligence, 2019. PDF DOI
Artikis A., Sergot M. and Paliouras G., An Event Calculus for Event Recognition.
In IEEE Transactions on Knowledge and Data Engineering (TKDE), 27(4), 895–908, 2015. PDF BibTeX DOI
Artikis A., Sergot M. and Paliouras G., A Logic Programming Approach to Activity Recognition.
In 2nd ACM International Workshop on Events in Multimedia (EIMM), pp. 3–8, 2010. PDF BibTeX DOI
Inductive Logic Programming-based Online Learning
OLED is an online learner constructing logic programs expressing Event Calculus patterns, that may be subsequently used for event recognition. OLED extends Inductive Logic Programming techniques for highly efficient learning over data streams. The videos below display event recognition when using the patterns for ‘meeting’ (two people are meeting) and ‘moving’ (two people are moving together), as constructed by OLED.
Katzouris N., Artikis A. and Paliouras G., Parallel Online Event Calculus Learning for Complex Event Recognition.
In Future Generation Computer Systems, 94, pp. 468–478, 2019. PDF BibTeX DOI
Katzouris N., Michelioudakis E., Artikis A. and Paliouras G., Online Learning of Weighted Relational Rules for Complex Event Recognition.
In European Conference on Machine Learning (ECML-PKDD), pp. 396–413, 2018. PDF Slides BibTeX DOI
Katzouris N., Artikis A., Paliouras G., Online Learning of Event Definitions.
In Theory and Practice of Logic Programming (TPLP), 16(5-6), pp. 817–833, 2016. PDF BibTeX DOI
Markov Logic-based Online Learning
OSLα is an online learner constructing Event Calculus patterns in Markov Logic, that may be subsequently used for (probabilistic) event recognition. The videos below illustrate the use of OSLα, by displaying event recognition when using the patterns for ‘meeting’ (two people are meeting) and ‘moving’ (two people are moving together), as constructed by OSLα.