Overview

Research Topics

Research Projects

Infrastructure

Scientific Contributions

Research Prototypes

Multimedia, Knowledge and Web Technologies Research Area

The area addresses a number of research topics that span the media/content lifecycle, namely:

• Content Authoring, Creation, Editing

  Defining models and tools for semantic-driven interactive storytelling and advanced template-based and collaborative authoring and editing of multimedia content, in many cases from multi-disciplinary teams, aiming at appropriate content creation and modeling in the context of a diverse number of application scenarios from interactive TV and games to learning modules and social media. Scene and multimedia description languages (VRML, X3D, SMIL, MPEG-4 XMT, etc.) are utilized to automate the generation of complex multimedia presentations.
  Our long-term experience in multimedia authoring and editing tools and environments, 2D/3D modeling and rendering tools, multimedia design, graphics and interactive content/multimedia presentations authoring provides us with the necessary skills to address research and innovate in this topic.
  One area of recent research addresses the semi-automated enrichment of audiovisual content with synchronized graphic objects placed in space and time at dynamically estimated positions. Where needed, such graphic objects are converted to interactive elements in the final multimedia presentation.

• Content Analysis and Information Extraction, Content Description and Annotation
  Devising algorithms and methodologies in order to automate the process of extracting information and discovering knowledge from multimedia and other content resources either published on the Web or stored in archives, and in that way efficiently describing them for further storage, retrieval, filtering and intelligent access, converting them to intelligent media.

Research in this area is undertaken in the following fields:

• Multimedia and other types of data pre-processing, spatio-temporal segmentation, normalization and feature extraction, optimized for the annotation/search field in question.
  o    Definition of metadata models based on multimedia and web standards for efficient semi-automated content description and tagging.
  o    Data mining and pattern classification algorithms for hidden information extraction towards medium level knowledge discovery.
  o    Investigation on early and late fusion algorithms to optimally integrate single modality or classifier on same topic results.
  o    Definition of multimedia and domain ontologies in order to represent knowledge either in data or in the domain in question using knowledge representation mechanisms.
  o    Complex knowledge discovery and information extraction (semantics) through the integration of logic-based reasoning and ontologies and probabilistic artificial intelligence algorithms.
  One area of recent research addresses the automated intelligent description and annotation of audio-visual data as to the existence of violent scenes, to further enable movie filtering for parental control purposes, defining violence and multimedia domain ontologies, relying on multimodal interaction from single modalities results, exploiting spatio-temporal relationships, and introducing advanced probabilistic reasoning algorithms for the identification of complex violent scenes.
  ■    Content Management and Archiving
  Digitization, restoration and quality enhancement of content for storage purposes, definition of content scalable representations and efficient spatio-temporal indexing structures for multimedia content to enable their fast search and browsing, query languages such as XQuery and SPARQL and metadata databases.
  In this context, a music data repository has been recently developed based on research conducted in the POLYMNIA project, in collaboration with the Cultural and Educational Technology Institute (CETI), that allowed efficient feature extraction, automated MPEG-7 based metadata generation and storage in open source XML database for the purposes of archiving and easily retrieving traditional Greek music.
  ■    Content Protection and Applications
  Major research interest is observed in digital rights management issues and protection of content copyrights, especially with the easy copying and distribution of digital data. In this field, our research focuses on investigating:
  o    Novel watermarking methodologies, deploying unique chaotic watermarks and transform embedding domains, for the copyright protection of multimedia data, robust against many common or malicious attacks, such as compression or cropping.
  o    Novel fingerprinting methodologies for usage tracing of data, robust after a series of transformation and compression steps maintaining a quality level.
  o    Tamper-proofing algorithms to detect illegal data modifications, by using fragile watermarks that are easily altered after modifications are introduced.
  The activity is not limited in the field of content protection, but in collaboration with members of other research areas at AIT, is extended to software protection from hacking and illegal modifications (mainly games), as well as to algorithms that alert fraud cases in online games.
  ■    Content Packaging and Delivery
  Efficient algorithms for cross-media configuration and packaging for optimal delivery, such as:
  o    Network monitoring mechanisms to extract dynamic network contextual information.
  o    Investigation of network adaptation methodologies such as rate control and advanced/scalable coding algorithms that lead to optimized content delivery enabling high user perceived content quality, in contexts of differentiating network contexts. Quality metrics are defined to determine the level of user perceived quality (QoP: quality of perception).
  o    Investigation of error concealment algorithms to account for the deterioration of content perceived quality due to transmission errors in compressed data.
  ■    Content Access and Consumption, Information Presentation and Visualization
  Optimizing user access to content, from anywhere, anytime, using any type of device. Research activities have targeted to different interpretations of optimal user access:
  o    Efficient and fast search, navigation and browsing of multimedia resources on the web or on dedicated media archives. Design and development of advanced search engines, media portals, and information systems. Specifically, research activities include:
  o    Video shot segmentation, shot/scene classification and summarization. Similar techniques are applied to image and auditory data in order to automatically classify them to general categories. Pattern classification and artificial intelligence algorithms are devised per case for this purpose.
  o    Multimedia indexing techniques to allow fast access and thus navigation to the points of interest.
  o    Information Retrieval and Data Mining algorithms, focusing on multimedia data that increase the algorithm’s complexity due to their multi-dimensionality and inherent multimodal dependencies.
  o    Early and late fusion algorithms to either combine multimodal features or increase the accuracy of identification results by relying to the outputs of many classifiers used to deduce the same outcome.
  o    Efficient matching algorithms to match metadata descriptions and/or features and thus efficiently retrieve results optimal to user queries or interactions.
  o    Investigation of user profiling methodologies per application domain, automating the process where possible, as well as personalization, targeting and filtering algorithms to automatically adapt and present content to the user based on his personal preferences or habits. Extension of such methodologies to account for group-based personalization and content adaptation as well as for device specific adaptation.
  o    Advanced information visualization and navigation patterns:
  o    Deployment of efficient data mining algorithms to categorize database entries and devise new visualization spaces revealing their information entropy.
  o    Advancement of existing ranking methods, to optimally present relevant to the user query results.
  The above research activities in combination with web technologies introduce further novel topics of research in a number of application areas, such as:
  ■    Service-Oriented Architectures and the Enterprise
  Research topics: (Semantic) web services, inter/intra – enterprise systems, workflow management, interoperability.
  ■    E-Learning - Edutainment
  Research topics:
  o    Infrastructural Innovation: GRID, P2P, Semantic Grid.
  o    Technological and pedagogical innovation: learning content management systems, virtual scientific experiments, virtual labs/classes, experiential-based learning, blended learning, advanced e-learning paradigms, gaming and learning, collaboration spaces, virtual communities, learning design, e-assessment, e-certification, etc.
  ■    E-Commerce, E-Business
  Research topics: business intelligence, enterprise systems, collaborative commerce – environments.
  ■    Digital entertainment and Serious games
  Research topics: gaming applications and simulations, dynamic simulations, adaptable games, multi-player/online gaming.
  Bearing in mind that for the interoperability and inter-working of distributed applications and services, as in the case of the web, the know-how and contribution, where possible, to relevant multimedia and web standards of the research area presents an important factor for sustainable research results:
  o    W3C Specifications: XML Technologies, RDF/RDFS, OWL, SWRl, SPARQL, SMIL, SVG, Video on the Web, Web services/WSDL/SOAP, HTML/XHTML/CSS, etc.
  o    E-learning relevant standards: SCORM, IEEE LOM.
  o    Metadata and other types of standards: Dublin Core, MPEG standards (2/4/7/21), TV Anytime specifications.