Ontological Reasoning

The immense and constantly growing amount of video resources requires efficient automated processing mechanisms, which is, however, a real challenge due to the Semantic Gap between low-level features automatically extracted and feature statistics and aggregates computed from audio and video signals, and what humans can comprehend based on acquired knowledge and years of experience. Automatically extractable descriptors, such as dominant colour and motion trajectory, have application potential in machine learning-based classification, but are limited in scene understanding, because they do not directly correspond to sophisticated human-interpretable concepts, such as depicted concepts and video events.

One of the main approaches to narrowing the Semantic Gap is to complement feature extraction and analysis with machine-interpretable background knowledge formalized using general, spatial, temporal, and fuzzy description logics and rule-based mechanisms, and implemented in ontology languages such as the Web Ontology Language (OWL) and rule languages such as SWRL. The corresponding structured annotations enable a variety of inference tasks suitable for the automated interpretation of images, 3D models, audio contents, and video scenes. They can be efficiently queried both manually and programmatically using the powerful query language SPARQL.

The formal representation of, and reasoning over, multimedia content descriptions, together with the semantic enrichment of multimedia resources with Linked Data, can be implemented in intelligent applications, such as video understanding, content-based video indexing and retrieval, automated subtitle generation, video surveillance, clinical decision support, and automated music and movie recommendation engines.

This research program focuses on the standardisation of structured spatiotemporal annotations, the development of Linked Data-powered hypervideo applications, and ontology-based video retrieval via spatiotemporal reasoning and information fusion.

Selected Publications

  • Sikos, L.F. and Powers, D.M. (2015). Knowledge-Driven Video Information Retrieval with LOD: From Semi-Structured to Structured Video Metadata. In Balog, K., Dalton, J., Doucet, A., Ibrahim, Y., ed. Proceedings of the Eighth Workshop on Exploiting Semantic Annotations in Information Retrieval. New York: ACM. 24th ACM International Conference on Information and Knowledge Management. Melbourne. Oct 2015.
  • Sikos, L. F. Ontology-Based Structured Video Annotation for Content-Based Video Retrieval via Spatiotemporal Reasoning. In: Kwa?nicka, H., Jain, L. C. (eds.) Bridging the Semantic Gap in Image and Video Analysis. Intelligent Systems Reference Library. Springer International Publishing, Cham, Switzerland, 2017
  • Sikos, L. F. Rich Semantics for Interactive 3D Models of Cultural Artifacts. In: Garoufallou, E., Subirats, I., Stellato, A., Greenberg, J. (eds.) Metadata and Semantics Research, pp. 169–180. Springer International Publishing, Cham, Switzerland, 2016. DOI: 10.1007/978-3-319-49157-8_14
  • Sikos, L. F. RDF-Powered Semantic Video Annotation Tools with Concept Mapping to Linked Data for Next-Generation Video Indexing. Multimedia Tools and Applications, Springer US, New York, USA, 2016. DOI: 10.1007/s11042-016-3705-7
  • Sikos, L. F. A Novel Approach to Multimedia Ontology Engineering for Automated Reasoning over Audiovisual LOD Datasets. In: Nguyen, N. T., Trawi?ski, B., Fujita, H., Hong, T.-P. (eds.) Intelligent Information and Database Systems, pp. 3–12. Springer Berlin Heidelberg, Heidelberg, Germany, 2016. DOI: 10.1007/978-3-662-49381-6_1

Featured publications

monograph and textbooks

  • Sikos, L. F. Description Logics in Multimedia Reasoning. Springer International Publishing, Cham, Switzerland, 2017, ISBN 978-3-319-54065-8, DOI: 10.1007/978-3-319-54066-5
  • Sikos, L. F. Mastering Structured Data on the Semantic Web: From HTML5 Microdata to Linked Open Data. Apress, New York, USA, 2015, ISBN 978-1-4842-1050-5, DOI: 10.1007/978-1-4842-1049-9
  • Sikos, L. F. Web Standards: Mastering HTML5, CSS3, and XML (Second Edition). Apress, New York, USA, 2014, ISBN-10: 1484208846, ISBN-13: 978-1-484208-84-7, DOI: 10.1007/978-1-4842-0883-0


People Involved

Dr Aaron Ceglar - Research Fellow
Dr Denise de Vries - Principal Investigator
Dr Carl Mooney - Principal Investigator
Prof. John Roddick - Principal Investigator
Prof. David Powers - Principal Investigator
Dr Leslie Sikos - Postdoctoral Researcher and Contact Person

Contributor to

W3C https://www.w3.org

Web3D Consortium https://www.web3d.org