Multimodal Signal Analytics

Humans can easily recognize familiar objects they see, or the sound of their favorite instruments in an orchestral recording, but they can easily spend minutes trying to multiply two numbers with a few decimal digits.

Machines that we can afford to buy for our homes on the other hand can multiply more than 10 trillion floating-point numbers in a second, yet encounter severe difficulties in familiarizing themselves with objects they sense in order to recognize them later.

In the Multimodal Signal Analytics group, we build systems that observe and understand their surroundings in an attempt to offer services to their inhabitants. To this end, we write software for a variety of applications, from the device level up to web and mobile applications.

To build systems that offer useful serviced by understanding their surroundings, we write software that on one hand implements mathematical algorithms, while on the other hand interfaces with devices.

Our hardware is:

  • Sensors (cameras, microphones, wearables, domotics devices) to capture the environment. Sometimes our sensors reside on smartphones
  • Actuators to attempt to interfere with the environment
  • Robots, comprising sensors and actuators, together with motorized parts

To interface with the hardware, we also build middleware implementing communication standards like USB, Wi-Fi, ZigBee or Bluetooth.

Since we deal with signals received from the sensors and sent to the actuators, our software meets mathematics by implementing signal processing algorithms. Signal processing, a domain of applied mathematics that attempts to extract meaning out of signals, is the core of our business.

We build:

  • Detectors, to locate the presence of signals of interest (e.g. human faces in images or human speech in audio)
  • Trackers, to follow detected objects of interest across time and space (e.g. cars from a video or a human speaker from a multi-microphone audio recording)
  • Classifiers, to recognize the specifics of the object (e.g. the identity or the gender of a tracked face, the emotion or the spoken words of a speaker)

Our signal processing algorithms are mainly built in C/C++, but we also utilize Python, Java, and MATLAB for prototyping.

Our signal processing algorithms, together with the sensors feeding them with signals (the data), form the perceptual components we use to sense and understand the environment.

Metadata that comes from a single perceptual component, or represents a short time span is not enough to understand situations and act upon them. For this reason, we store the metadata in databases. We employ both SQL and NoSQL databases depending on the application at hand.

Our reasoners are another type of software we build (mainly in Java) to reason for situations in our perceived environments. E.g. many faces detected in a meeting room, each speaking in turn, allows us to deduce that a meeting is taking place, while abrupt acceleration from a wearable sensor, followed by no steps for some time, allows us to detect a falling accident.

Understanding situations like these can trigger services, where information is presented to the users in the perceived space, or outsiders. E.g. automated transcripts of the aforementioned meeting can become available to the participants, or to remote interested parties. An automated alert can be sent to the next of kin of the fallen person. We use Java (or PHP, NodeJS) and Javascript (ReactJS, AngularJS, jQuery) respectively to write the backend and the frontend of our services.

Computer vision

Tracking (visual and multi-modal, two-dimensional and three-dimensional)

Face & facial features tracking:

Multi-cue face tracking by fusing colour, foreground and face detection:

Multi-camera 3D tracking using motion and feature points:

3D head tracking fusing face detection and motion information from four stationary cameras:

3D tracking of location and gaze direction by fusing motion and face detections from four stationary cameras:

Outdoors pedestrian and vehicle tracking:

Audio-visual speaker tracking:

Depth estimation using a fixed monocular camera:

Detection (faces, bodies or objects)

Demonstration of YOLO3 detections on the Dunkirk trailer:

Human-Computer Interaction

VR demo with real and digital objects interacting on a tabletop:

Gesture-enabled map exploration:

Nod and gesture recognition using motion information:

Pointing demo where a rod is tracked in 3D using a pair of cameras. The direction of the rod is projected towards the wall, as shown in another pair of cameras:

Finger tracking for multi-touch surfaces:

Medical image processing

Create a large image by stitching together the frames received from a microscope:

Pattern classification

Face identity, expression, gender and age recognition

Image of classification workflow:

Face tracking and analytics:

Applications in Smart Environments

m-health and e-health solutions

Games Of Active Life Android app:

Fall risk assessment web app:

MyKitchenManager App Demo:

Smart working and living environments

Visual scene analysis for crowds and vehicles in different zones: the streets, the sidewalks and the square:

Analysis of crowds in different zones: the street (top) and the plaza (bottom):

Personalised media broadcasting

Video processing in sports recordings:

Motion analysis of sports recordings by projectively-matched frame-by-frame difference:

Retail applications

Hotspots detection via a floorplan occupancy map:

Track people moving up and down at escalators using Kinect:

 

 

At the Multimodal Signal Analytics group, we offer customisable solutions to detect, track and classify different objects from a variety of sensors, eventually building services for any application. We implement the necessary algorithms in real-time software, but also have full-stack developers to produce applications for desktop, web and mobile platforms.

Our products have been demonstrated at exhibitions and technology shows, evaluated at international evaluation campaigns and featured on Greek national TV, radio and newspapers.

Our research and development efforts have been co-founded by many European, national and industrial projects, and have resulted in publications of books, patents, and scientific articles in journals and conferences.

Projects

  1. Sample Deposit Station. Work subcontracted by Ascend Diagnostics, involving the development of software (front and back end) that controls the clinical process around a hardware for mass spectroscopy (11/2018 – to date).
  2. Whole Slide Image Scanner. Work subcontracted by DeepMed I/O, involving the development of software that controls an automatic Whole Slide Image Scanner and performs Medical image stitching for the production of Whole Slide Images for use in diagnostic purposes (9/2018 – to date).
  3. Work subcontracted by Rokk Media, involving the development of a smartphone application for displaying measurements from sensors via BLE and configuring those sensors. Both an Android and an iOS native application are developed (7/2018 – to date).
  4. Ticket Design Tool. Work subcontracted by Intralot, involving the creation of a CAD tool (front and back end) to be used in designing lottery tickets, while enforcing the different families of layout rules of Intralot (3/2018 – to date).
  5. GOAL (Games Of Active Life, H2020-ICT-2016-1, http://www.goal-h2020.eu/). Provide a platform for games & apps, aiming to motivate a socially and physically active lifestyle. Technical coordinator (10/2016 – to date).
  6. Camera Reader. Work subcontracted by Intralot, involving the creation of a system (HW and SW) for reading lottery tickets by a camera. Work resulted into the system being sold world-wide and multiple patents being granted. Prototyping of the algorithms is done in MATLAB, while the implementation is done in C for an ARM processor (5/2006 – to date).
  7. Work subcontracted by Rokk Media, involving the full-stack development of a handset purchase via loans management system. The backend is built in Java using the Spring framework and PostgreSQL Database, while the frontend utilises HTML, CSS, Javascript and jQuery (2/2018 – 9/2018).
  8. Medical Image Stitching. Work subcontracted by DeepMed I/O, involving the development of an image stitching system for images from a microscope (2/2018 – 7/2018).
  9. Prizes (Management System for Prizes for the EP). Work subcontracted by Intrasoft International, involving the full-stack development of a prizes management system. The backend is built in Java using the Spring framework and Oracle Database, while the frontend utilises HTML, CSS, Javascript and jQuery (5/2017 – 3/2018).
  10. Ticket Tester. Work subcontracted by Intralot, involving the creation of a system for testing the results of printed elements in lottery tickets versus the expected elements from the game generator (5/2017 – 2/2018).
  11. Time Machine (Game for the EP). Work subcontracted by Intrasoft International, involving the development of a game (mixture of arcade and quiz), built in COCOS2d (1/2017 – 1/2018).
  12. Papyrus (Document Gateway for the HR of EP). Work subcontracted by Intrasoft International, involving the full-stack development of a document gateway. The backend is built in Java using the Spring framework and Oracle Database and WebCenter, while the frontend utilises HTML, CSS, Javascript and jQuery (4/2016 – 12/2017).
  13. VCT (Virtual Coaching Tool). Work subcontracted by Intrasoft International, involving the creation of custom media players in ActionScript and their integration in a web-based training platform for interpreters (12/2014 – 12/2016).
  14. OCTAVE (Objective Control for TAlker VErification, H2020-DS-2014-1, https://www.octave-project.eu/). Bringing environmental robustness and anti-spoofing methods to voice biometric authentication for secure access to smart services and critical infrastructures (6/2015 – 9/2017).
  15. eWALL (E-Wall for Active Long Living, FP7-610658, ewallproject.eu). Processing diverse sensors’ signals for user context extraction in an e-care home environment, representation of the resulting metadata and reasoning for care recipient’s state. Work-package 3 leader (11/2013 – 10/2016).
  16. ARGOS (Advanced pRotection of critical buildinGs by Overall anticipating System, FP7-SEC-2012-1, no. 313217, http://www.argos-project.eu/). Multimodal early warning security solution for critical infrastructures. Audio/visual signal processing for events extraction, prototyping in MATLAB and implementing in C++ (1/2014 – 2/2016).
  17. SMART (Search engine for MultimediA enviRonment generated contenT, FP7-ICT 287583, http://www.smartfp7.eu/). Synthesis of searchable events by fusion of metadata perceived from sensor networks, filtered from and social networks and retrieved from the linked data cloud. Prototyping carried out in MATLAB and implementing in C++. Work-package 4 leader (11/2011 – 11/2014).
  18. THIS (Transport Hub Intelligent video System, JLS/2009/CFP/CIPS “Prevention, Preparedness, and Consequence Management of Terrorism and other Security Related Risks”). Fusion and analysis of field video data to identify suspect individuals. Prototyping carried out in MATLAB and implementing in C++ (12/2009 – 12/2011).
  19. AGNES (User-sensitive Home-based Systems for Successful Ageing in a Networked Society, ALL-2008-1-014). Automated systems to fight social isolation of the elderly for extended independent living. Gesture recognition. Prototyping carried out in MATLAB and implementing in C++ (9/2009 – 9/2012).
  20. MEDUSA (Multi Sensor Data Fusion Grid for Urban Situational Awareness, European Defense Agency, A-0444-RT-GC JIP-FP Call 3 – Sensor Fusion). Video signal processing and sensor fusion. Prototyping carried out in MATLAB and implementing in C++ (9/2008 – 1/2012).
  21. My e-Director 2012 (Real-Time Context-Aware and Personalized Media Streaming Environments for Large Scale Broadcasting Applications, FP7-ICT 215248). Provide a unique interactive broadcasting service enabling end-users to select focal actors and points of interest within real-time broadcasted scenes. Prototyping carried out in MATLAB and implementing in C.Work-package 4 leader (2/2008 – 7/2011).
  22. HERMES (Cognitive Care and Guidance for Active Aging, FP7-ICT 216709, http://www.fp7-hermes.eu/). Provide an integrated approach to cognitive care, by an assistive technology that combines the functional skills of the older person to reduce age-related decline of cognitive capabilities and assist the user where necessary. Prototyping carried out in MATLAB and implementing in C. Work-package 4 leader (1/2008 – 7/2011).
  23. ΠΡΙΑΜΟΣ (GSRT), 3rd Framework, section 3.3. Computer vision and statistical signal processing for human-machine interfaces. Development carried out in MATLAB (6/2006 – 12/2007).
  24. PROSTO (GSRT). Teaching appropriately selected target groups on technology topics relating to managing transport traffic and vehicles (5/2004 – 5/2006).
  25. CHIL (Computers in the Human Interaction Loop, IST-2002-506909). Fade the computers into the background assisting human-to-human interaction. Involved in work-package 4, Who and Where (Computer Vision). Prototyping carried out in MATLAB and implementing in C (1/2004 – 8/2007).

Publications – Books

Authored

  1. Bourdopoulos, A. Pnevmatikakis, V. Anastassopoulos and T. Deliyannis, ‘Delta-Sigma Modulators, Modeling, Design and Applications’, World Scientific Publication Company, Imperial College Press, Sept. 2003.
  2. Talantzis, A. Pnevmatikakis and A.G. Constantinides, ‘Audio-Visual Person Tracking: A Practical Approach,’ World Scientific Publication Company, Imperial College Press, Dec. 2011.

Edited

  1. Boukis, A. Pnevmatikakis and L. Polymenakos (eds.), ‘Artificial Intelligence and Innovations 2007: From Theory to Applications’, Springer, Berlin Heidelberg, ISBN 978-0-387-74160-4, Sept. 2007.
  2. Tsekeridou, A. Pnevmatikakis, K. Wong, T. Tiropanis and R. Nakatsu (eds.), ‘3rd International Conference on Digital Interactive Media in Entertainment and Arts’, ISBN: 978-1-60558-248-1, Sept. 2008.

Contributed to

  1. Pnevmatikakis and L. Polymenakos, ‘Far-Field Multi-Camera Video-to-Video Face Recognition’, in K. Delac and M. Grgic (eds.), Face Recognition, I-Tech Education and Publishing, ISBN 978-3-902613-03-5.
  2. Pnevmatikakis, ‘Where and Who? Person Tracking and Recognition System’, in I. Maglogiannis, K. Karpouzis, M. Wallace and J. Soldatos (eds.), Emerging Artificial Intelligence Applications in Computer Engineering, IOS Press, ISBN 978-1-58603-780-2, Oct. 2007.
  3. Pnevmatikakis, H. K. Ekenel, C. Barras and J. Hernando, ‘Multimodal Person Identification’, in A. Waibel and R. Stiefelhagen (Eds.), ‘Computers in the Human Interaction Loop’, Springer, ISBN: 978-1-84882-053-1, 2009.
  4. Bernardin, R. Stiefelhagen, A. Pnevmatikakis, O. Lanz, A. Brutti, J. R. Casas and G. Potamianos, ‘Person Tracking’, in A. Waibel and R. Stiefelhagen (Eds.), ‘Computers in the Human Interaction Loop’, Springer, ISBN: 978-1-84882-053-1, 2009.
  5. Mihovska, A. Pnevmatikakis, S. Kyriazakos, K. Tonchev, R. Craciunescu, V. Poulkov, H. op den Akker and H. Hermens, “Integration of Sensing Devices and the Cloud for Innovative e-Health Applications” in F. J. Velez and F. D. Miyandoab (eds.), Wearable Technologies and Wireless Body Sensor Networks for Healthcare, IET, UK, 2018, accepted for publication.

Publications – Patents

  1. A. Pnevmatikakis and A. Diamandis, “Διαμόρφωση εντύπου πολλαπλών επιλογών αναγνώσιμου με ηλεκτρονικό μέσο και σύστημα και μέθοδος για την ερμηνεία κατ’ ελάχιστον μίας επιλογής του χρήστη”, Greek patent, no. 20080100525, Aug. 2008.
  2. Pnevmatikakis and A. Diamandis, “Machine-Readable form configuration and system and method for interpreting at least one user mark”, Patent Cooperation Treaty, no. PCT/US09/52032, July 2009.
  3. Pnevmatikakis and A. Diamandis, “System and method for interpreting at least one user mark on a machine- readable form configuration”, European patent no. 2151788 B1, Mar. 2011.
  4. Pnevmatikakis and A. Diamandis, “Machine-readable form configuration and system and method for interpreting at least one user mark”, U.S. patent no. 8587663 B2, Nov. 2013.
  5. Pnevmatikakis and A. Diamandis, “Machine-readable form configuration and system and method for interpreting at least one user mark”, US patent no. 9349064 B2, May 2016.
  6. Pnevmatikakis and A. Diamandis, “Machine-readable form configuration and system and method for interpreting at least one user mark”, European patent no. 2367142 B1, Oct. 2017.
  7. Pnevmatikakis and A. Diamandis, “Method and system for detecting user marks in a machine-readable form”, European patent no. 2565823 B1, Oct. 2017.

Publications – Journals

  1. F. Talantzis, D. B. Ward, “Robustness Of Multi-Channel Equalization In Acoustical Reverberant Environments,” Journal of Acoustical Society of America, vol. 114, no. 2, pp. 833-841, Aug. 2003.
  2. F. Talantzis, A. G. Constantinides, L. C. Polymenakos, “Estimation of Direction of Arrival Using Information Theory,” IEEE Signal Processing, vol. 12, no. 8, pp. 561 – 564, Aug. 2005.
  3. F. Talantzis, D. B. Ward, P. A. Naylor, “Performance Analysis of Dynamic Acoustic Source Separation In Reverberant Rooms,” IEEE Transactions on Audio, Speech and Language Processing, vol. 14, no. 4, pp. 1378 – 1390, July 2006.
  4. D. Mostefa, N. Moreau, K. Choukri, G. Potamianos, S. Chu, A. Tyagi, J. Casas, L. Christoforetti, F. Tobia, A. Pnevmatikakis, V. Mylonakis, F. Talantzis, S. Burger, R. Stiefelhagen, K. Bernardin and C. Rochet, ‘The CHIL Audiovisual Corpus for Lecture and Meeting Analysis inside Smart Rooms’, Journal on Language Resources and Evaluation, Vol. 41, No. 3-4, pp. 389-407, Springer, Dec. 2007.
  5. F. Talantzis, A. Pnevmatikakis, and A. G. Constantinides, ‘Audio-Visual Active Speaker Tracking In Cluttered Indoors Environments’, Journal of Systems, Man, and Cybernetics–Part B: Cybernetics, pp. 799-807, June 2008.
  6. A. Pnevmatikakis, F. Talantzis, J. Soldatos and L. Polymenakos, ‘Robust Multimodal Audio-Visual Processing for Advanced Context Awareness in Smart Spaces’, Personal and Ubiquitous Computing, Vol. 13, No. 1, pp. 3-14(12), Jan. 2009.
  7. A. Pnevmatikakis and L. Polymenakos, ‘Subclass Linear Discriminant Analysis for Video-Based Face Recognition’, Journal of Visual Communication and Image Representation, Vol. 20, Issue 8, pp. 543-551, Nov. 2009.
  8. F. Talantzis, A. G. Constantinides, “A Multi-Microphone Voice Activity Detection System Based on Mutual Information,” Journal of the Acoustical Engineering Society (AES), Vol. 57, No. 11, Nov. 2009.
  9. V. Theodoreli, Th. Petsatodis, J. Soldatos, F. Talantzis and A. Pnevmatikakis, “A Low-Cost Multi-Touch Surface Device supporting Effective Ergonomic Cognitive Training for the Elderly”, International Journal of Ambient Computing and Intelligence, Vol. 2, No. 3, pp. 50-62, July-Sept. 2010.
  10. F. Talantzis, “An Acoustic Source Localization and Tracking Framework Using Particle Filtering and Information Theory,” IEEE Transactions on Audio, Speech and Language Processing, Vol. 18, No. 7, pp. 1806 – 1817, Sep. 2010.
  11. Ch. Patrikakis, A. Pnevmatikakis, P. Chippendale, M. Nunes, R. Cruz, S. Poslad, Z. Wang, N. Papaoulakis, P. Papageorgiou, ‘Direct your personal coverage of large athletic events’, IEEE Multimedia, DOI 10.1109/MMUL.2010.69, Nov. 2010.
  12. T. Petsatodis, C. Boukis, F. Talantzis, Z. Tan, “Convex Combination of Multiple Statistical Models with application to VAD,” IEEE Transactions on Audio, Speech and Language Processing, Vol. 19, No. 8, pp. 2314-2327, Nov. 2011.
  13. T. Petsatodis, F. Talantzis, Z. Tan, “Exploring Super-Gaussianity towards robust information-theoretical time delay estimation,” Journal of the Acoustical Society of America (JASA), Vol. 133, No. 3, DOI: 10.1121/1.4776193, 2013.
  14. N. Katsarakis, A. Pnevmatikakis, Z.-H. Tan and R. Prasad, “Combination of Multiple Measurement Cues for Visual Face Tracking,” Wireless Personal Communications, special issue on “Intelligent Infrastructures & Beyond,” vol. 78, no. 3, pp. 1789-1810, Springer, October 2014.
  15. N. Katsarakis, A. Pnevmatikakis, Z.-H. Tan and R. Prasad, “Improved Gaussian Mixture Models for Adaptive Foreground Segmentation,” Wireless Personal Communications, special issue on “Current trends in information and communication technology,” vol. 87, no. 3, pp. 629-643, Springer, April 2016.
  16. G. Bardas, S. Astaras, S. Diamantas and A. Pnevmatikakis, “3D tracking and classification system using a monocular camera,” Wireless Personal Communications, vol. 92, no. 1, pp 63–85, Springer, Jan. 2017.
  17. A. Pnevmatikakis, “Recognising Daily Functioning Activities in Smart Homes,” Wireless Personal Communications, vol. 96, no. 3, pp 3639–3654, Springer, Oct. 2017.
  18. S. Kyriazakos, A. Pnevmatikakis et.al, “eWALL: An Open-Source Cloud-Based eHealth Platform for Creating Home Caring Environments for Older Adults Living with Chronic Diseases or Frailty,” Wireless Personal Communications, vol. 97, no. 2, pp 1835–1875, Springer, Nov. 2017.
  19. S. Astaras and A. Pnevmatikakis, “Visual detection of events of interest from urban activity,” Wireless Personal Communications, vol. 97, no. 2, pp 1877–1888, Springer, Nov. 2017.
  20. F. Talantzis, “Report on the real-time use of audio pre-processing technologies for access control in domestic environments,” IEEE Transactions on Systems, Man, and Cybernetics, Part B, 2019

Publications – Conferences

  1. F. Talantzis, D. B. Ward, “Multi-channel equalization in an acoustic reverberant environment: Establishment of robustness measures,” Institute of Acoustics Spring Conference, Salford, UK, March 2002.
  2. F. Talantzis and D. B. Ward, “Investigation of performance of acoustic arrays for equalization in a reverberant environment,” Proceedings of IEEE 14th Int. Conf. Digital Signal Processing, vol. I, pp. 247250, Santorini, Greece, July 2002.
  3. F. Talantzis, D. B. Ward, P. A. Naylor, “Expected Performance of a Family of Blind Source Separation Algorithms in a Reverberant Room,” Proceedings of IEEE ICASSP 2004, vol. 4, pp. 61-64, Montreal, Canada, May 2004.
  4. A. Pnevmatikakis and L. Polymenakos, ‘Comparison of Eigenface-Based Feature Vectors under Different Impairments’, 17th Int. Conf. Pattern Recognition 2004, vol. 1, pp. 296-300, Cambridge, UK, Aug. 2004.
  5. J. Soldatos, L. Polymenakos, A. Pnevmatikakis, F. Talantzis, K. Stamatis and M. Carras, ‘Perceptual Interfaces and Distributed Agents supporting Ubiquitous Computing Services’, Eurescom 2005.
  6. S. Azodolmolky, N. Dimakis, V. Mylonakis, G. Souretis, J. Soldatos, A. Pnevmatikakis and L. Polymenakos, ‘Middleware for In-door Ambient Intelligence: The PolyOmaton System’, 2nd Next Generation Networking Middleware Workshop (NGNM05), Waterloo, Canada, May 2005.
  7. A. Pnevmatikakis and L. Polymenakos, ‘An Automatic Face Detection and Recognition System for Video Streams’ 2nd Joint Workshop on Multi-Modal Interaction and Related Machine Learning Algorithms, Edinburgh, UK, July 2005.
  8. A. Stergiou, A. Pnevmatikakis and L. Polymenakos, ‘Audio/Visual Person Identification’ 2nd Joint Workshop on Multi-Modal Interaction and Related Machine Learning Algorithms, Edinburgh, UK, July 2005.
  9. A. Pnevmatikakis and L. Polymenakos, ‘A testing methodology for face recognition algorithms’, in S. Renals and S. Bengio (Eds.): MLMI 2005, Lecture Notes in Computer Science 3869, pp. 218-229, Springer-Verlag, Berlin Heidelberg, 2006.
  10. V. Mylonakis, F. Talantzis and L. Polymenakos, ‘Real-Time Audio Source Localization Using Information Theory’, in S. Renals, S. Bengio and J. Fiscus (Eds.): MLMI 2006, Lecture Notes in Computer Science, Springer-Verlag, Berlin Heidelberg, 2006.
  11. A. Stergiou, A. Pnevmatikakis and L. Polymenakos, ‘EBGM Vs Subspace Projection for Face Recognition’ International Conference on Computer Vision Theory and Applications, Setúbal, Portugal, Vol. 2, pp. 131-137, Feb. 2006.
  12. H. Ekenel and A. Pnevmatikakis, ‘Video-Based Face Recognition Evaluation in the CHIL Project – Run 1’, Face and Gesture Recognition 2006, Southampton, UK, pp. 85-90, Apr. 2006.
  13. A. Pnevmatikakis and L. Polymenakos, ‘2D Multi-Person Tracking Using Kalman Filtering and Adaptive Background Learning in a Feedback Loop’, in R. Stiefelhagen and J. Garofolo (eds.): CLEAR 2006, Lecture Notes in Computer Science 4122, Springer-Verlag, Berlin Heidelberg, 2006.
  14. N. Katsarakis, G. Souretis, F. Talantzis, A. Pnevmatikakis and L. Polymenakos, ‘3D Audiovisual Person Tracking Using Kalman Filtering and Information Theory’, in R. Stiefelhagen and J. Garofolo (eds.): CLEAR 2006, Lecture Notes in Computer Science 4122, Springer-Verlag, Berlin Heidelberg, 2006.
  15. A. Stergiou, A. Pnevmatikakis and L. Polymenakos, ‘A Decision Fusion System across Time and Classifiers for Audio-visual Person Identification’, in R. Stiefelhagen and J. Garofolo (eds.): CLEAR 2006, Lecture Notes in Computer Science 4122, Springer-Verlag, Berlin Heidelberg, 2006.
  16. A. Pnevmatikakis, L. Polymenakos and V. Mylonakis, ‘The AIT Outdoors Tracking System for Pedestrians and Vehicles’, in R. Stiefelhagen and J. Garofolo (eds.): CLEAR 2006, Lecture Notes in Computer Science 4122, Springer-Verlag, Berlin Heidelberg, 2006.
  17. E. Rentzeperis, A. Stergiou, C. Boukis, G. Souretis, A. Pnevmatikakis and L. Polymenakos, ‘An Adaptive Speech Activity Detector Based on Signal Energy and LDA’, 3rd Joint Workshop on Multi-Modal Interaction and Related Machine Learning Algorithms, 2006.
  18. A. Pnevmatikakis and L. Polymenakos, ‘Kalman Tracking with Target Feedback on Adaptive Background Learning’, in S. Renals, S. Bengio and J. Fiscus (Eds.): MLMI 2006, Lecture Notes in Computer Science 4299, Springer-Verlag, Berlin Heidelberg, pp. 114-122, 2006.
  19. E. Rentzeperis, A. Stergiou, C. Boukis, A. Pnevmatikakis and L. Polymenakos, ‘The 2006 Athens Information Technology Speech Activity Detection and Speaker Diarizaton Systems’, in R. Stiefelhagen and J. Garofolo (eds.), RT’06, Lecture Notes in Computer Science, Springer-Verlag, Berlin Heidelberg.
  20. E. Rentzeperis, A. Stergiou, A. Pnevmatikakis and L. Polymenakos, ‘Impact of Face Registration Errors on Recognition’, in I. Maglogiannis, K. Karpouzis and M. Bramer (eds.), Artificial Intelligence Applications and Innovations (AIAI06), Springer, Berlin Heidelberg, pp. 187-194, June 2006.
  21. A. Pnevmatikakis, F. Talantzis, J. Soldatos and L. Polymenakos, ‘Robust Multimodal Audio-Visual Processing for Advanced Context Awareness in Smart Spaces’, in I. Maglogiannis, K. Karpouzis and M. Bramer (eds.), Artificial Intelligence Applications and Innovations (AIAI06), Springer, Berlin Heidelberg, pp. 290-301, June 2006.
  22. A. Stergiou, A. Pnevmatikakis and L. Polymenakos, ‘Enhancing the Performance of a GMM-based Speaker Identification System in a Multi-Microphone Setup’, InrterSpeech 2006, Pitsburgh, USA, Sept. 2006.
  23. F. Talantzis, A. G. Constantinides and L. Polymenakos, ‘Using a differential microphone array to estimate the direction of arrival of two acoustic sources’, InrterSpeech 2006, Pitsburgh, USA, Sept. 2006.
  24. F. Talantzis, A. Pnevmatikakis and L. Polymenakos, ‘An Automatic Face Detection and Recognition System for Video Streams’, IEEE International Workshop on Multimedia Signal Processing (MMSP 2006), Victoria, Canada, Oct. 2006.
  25. A. Pnevmatikakis and L. Polymenakos, ‘Robust Estimation of Background for Fixed Cameras’, 15th International Conference on Computing (CIC2006), Mexico City, Mexico, Nov. 2006 (Best paper award, 3rd place).
  26. A. Stergiou, G. Karame, A. Pnevmatikakis and L. Polymenakos, ‘The AIT face tracker for VACE multisite meeting recordings’, CLEAR 2007, Baltimore, USA, May 2007.
  27. F. Talantzis, A. G. Constantinides, C. Boukis, “The robustness effect of acoustic source localization on blind source separation and deconvolution systems”, 15th International Conference on Digital Signal Processing DSP07, June 2007, Cardiff.
  28. F. Talantzis, L. Polymenakos, ”Robustness of Non-Exact Multi-Channel Equalization in Reverberant Environments”, Proceedings of Artificial Intelligence Applications & Innovations (AIAI), 2007.
  29. F. Talantzis, A. G. Constantinides,”The effect of microphone array geometry on acoustic source localization”, International Congress on Noise Control Engineering INTERNOISE07, Istanbul, Turkey, Aug. 2007.
  30. G. Karame, A. Stergiou, N. Katsarakis, P. Papageorgiou and A. Pnevmatikakis, ‘2D and 3D Face Localization for Complex Scenes’, IEEE Int. Conf. on Advanced Video and Signal based Surveillance (AVSS 2007), London, UK, Sept. 2007.
  31. N. Katsarakis, A. Pnevmatikakis and M. Nechyba, ‘3D Tracking of Multiple People Using Their 2D Face Locations’, in C. Boukis, A. Pnevmatikakis and L. Polymenakos (eds.), Artificial Intelligence and Innovations 2007: From Theory to Applications (AIAI07), Springer, Berlin Heidelberg, pp. 365-374, Sept. 2007.
  32. E. Rentzeperis, C. Boukis, A. Pnevmatikakis and L. Polymenakos, ‘Combining Finite State Machines and LDA for Voice Activity Detection’, in C. Boukis, A. Pnevmatikakis and L. Polymenakos (eds.), Artificial Intelligence and Innovations 2007: From Theory to Applications (AIAI07), Springer, Berlin Heidelberg, pp. 323-330, Sept. 2007.
  33. F. Talantzis, L. Polymenakos, ”Robustness of Non-Exact Multi-Channel Equalization in Reverberant Environments”, Proceedings of Artificial Intelligence Applications & Innovations (AIAI), Athens, Greece, Sep. 2007.
  34. A. Stergiou, G. Karame, A. Pnevmatikakis and L. Polymenakos, ‘The AIT 2D face detection and tracking system for CLEAR 2007’, in R. Stiefelhagen, R. Bowers and J. Fiscus (eds.): Multimodal Technologies for Perception of Humans (CLEAR 2007), Lecture Notes in Computer Science 4625, Springer-Verlag, Berlin Heidelberg, pp. 113-125, June 2008.
  35. N. Katsarakis, F. Talantzis, A. Pnevmatikakis and L. Polymenakos, ‘The AIT 3D Audio / Visual Person Tracker for CLEAR 2007’, in R. Stiefelhagen, R. Bowers and J. Fiscus (eds.): Multimodal Technologies for Perception of Humans (CLEAR 2007), Lecture Notes in Computer Science 4625, Springer-Verlag, Berlin Heidelberg, pp. 35-46, June 2008.
  36. A. Stergiou, A. Pnevmatikakis and L. Polymenakos, ‘The AIT multimodal person identification system for CLEAR 2007’, in R. Stiefelhagen, R. Bowers and J. Fiscus (eds.): Multimodal Technologies for Perception of Humans (CLEAR 2007), Lecture Notes in Computer Science 4625, Springer-Verlag, Berlin Heidelberg, pp. 221-232, June 2008.
  37. A. Stergiou, A. Pnevmatikakis and L. Polymenakos, ‘The AIT outdoor tracker for vehicles and pedestrians in CLEAR 2007’, in R. Stiefelhagen, R. Bowers and J. Fiscus (eds.): Multimodal Technologies for Perception of Humans (CLEAR 2007), Lecture Notes in Computer Science 4625, Springer-Verlag, Berlin Heidelberg, pp. 148-159, June 2008.
  38. V. Mylonakis, J. Soldatos, A. Pnevmatikakis, L. Polymenakos, A. Sorin and H. Aronowitz, ‘Using Robust Audio and Video Processing Technologies to Alleviate the Elderly Cognitive Decline’, Pervasive Technologies Related to Assistive Environments (PETRA 2008), Corfu, Greece, July 2008.
  39. A. Anagnostopoulos and A. Pnevmatikakis, ‘A realtime mixed reality system for seamless interaction between real and virtual objects’, Digital Interactive Media in Arts and Entertainment (DIMEA 2008), Athens, Greece, Sept. 2008.
  40. N. Katsarakis, A. Pnevmatikakis and J. Soldatos, ‘Person Tracking for Ambient Camera Selection in Complex Sports Environments’, Digital Interactive Media in Arts and Entertainment (DIMEA 2008), Athens, Greece, Sept. 2008.
  41. N. Papaoulakis, N. Doulamis, Ch. Patrikakis, J. Soldatos, A. Pnevmatikakis and E. Protonotarios, ‘Real-Time Video Analysis and Personalized Media Streaming Environments for Large Scale Athletic Events’, ACM Multimedia 2008, Vancouver, BC, Canada, Oct. 2008.
  42. A. Stergiou, N. Katsarakis, A. Pnevmatikakis and L. Polymenakos, ‘Detecting Single-Actor Events in Video Streams for TRECVid 2008’, TRECVid 2008 workshop, USA, Nov. 2008.
  43. Ch. Doukas, I. Maglogiannis, N. Katsarakis and A. Pnevmatikakis, “Enhanced Human Body Fall Detection utilizing Advanced Classification of Video and Motion Perceptual Components”, Artificial Intelligence Applications and Innovations (AIAI09), April 2009.
  44. F. Talantzis, A. Constantinides, “Using information theory to detect voice activity”, Proceedings of IEEE ICASSP 2009, Taipei, Taiwan, April 2009.
  45. D. Facal, M.F. Gonzalez, F. Talantzis, “Cognitive Games for Healthy Elderly People in a Multitouch Screen”, Proceedings of International Congress on Digital Homes, Robotics and Telecare for All, Barcelona, Spain, May 2009.
  46. O. Alrabadi, F. Talantzis, A. Constantinides, “Locating an Acoustic Source Using a Mutual Information Beamformer”, Proceedings of Artificial Intelligence Applications & Innovations (AIAI), 2009.
  47. N. Katsarakis and A. Pnevmatikakis, ‘Event Detection in Athletics for Personalized Sports Content Delivery’, 10th International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS 2009), London, UK, May 2009.
  48. S. Poslad, A. Pnevmatikakis, M. Nunes, E. Garrido Ostermann, P. Chippendale, P. Brightwell and Ch. Patrikakis, ‘Directing Your Own Live and Interactive Sports Channel’, 10th International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS 2009), London, UK, May 2009.
  49. Ch. Doukas, I. Maglogiannis, A.N. Rouskas and A. Pnevmatikakis, “Emergency incidents detection in assisted living environments utilizing sound and visual perceptual components,” Pervasive Technologies Related to Assistive Environments (PETRA 2009), Corfu, Greece, June 2009.
  50. N. Katsarakis and A. Pnevmatikakis, ‘Face Validation Using 3D Information from Single Calibrated Camera’, DSP 2009, Santorini, Greece, July 2009.
  51. Th. Petsatodis, F. Talantzis, A. Pnevmatikakis and U. Diaz, ‘Interactive surfaces for enhanced cognitive care’, DSP 2009, Santorini, Greece, July 2009.
  52. Th. Petsatodis, Ch. Boukis and A. Pnevmatikakis, ‘Voice Activity Detection Using Audio-Visual Information’, DSP 2009, Santorini, Greece, July 2009.
  53. Th. Petsatodis, J. Soldatos, A. Pnevmatikakis and F. Talantzis, ‘Multi-Touch Surfaces as Motivating and Ergonomic Environments for Elderly Cognitive Training’, workshop on Designing Ambient Interactions for Older Users, AMI 2009, Salzburg, Nov. 2009.
  54. D. Facal, M.-F. González, A. Navarro, C. Oppenauer, A. Geven, S. Prost, M. Tscheligi, T. Petsatodis, A. Pnevmatikakis, V. Theodoreli, E. Urdaneta and J. Yanguas, “Qualitative assessment of computerized-cognitive games for elderly people”, Abstracts of the IPA international meeting. In R. Mateos, K. Engedal and M. Franco (eds), pp. 294-295, 2010. ISBN 978-84-9887 -529 – 4.
  55. A. Pnevmatikakis and F. Talantzis, “Person tracking in enhanced cognitive care: A particle filtering approach”, special session on Person tracking for assistive working and living environments, EUSIPCO 2010, Aalborg, Denmark, Aug. 2010.
  56. M. Andersen, R. Andersen, N. Katsarakis, A. Pnevmatikakis and Z.-H. Tan, “Three-dimensional adaptive sensing of people in a multi-camera setup”, special session on Person tracking for assistive working and living environments, EUSIPCO 2010, Aalborg, Denmark, Aug. 2010.
  57. A. Pnevmatikakis, N. Katsarakis, P. Chippendale, Cl. Andreatta, S. Messelodi, C. Modena and F. Tobia, “Tracking for Context Extraction in Athletic Events”, International Workshop on Social, Adaptive and Personalized Multimedia Interaction and Access (SAPMIA 2010), ACM Multimedia, Florence, Italy, Oct. 2010.
  58. P. Chippendale, A. Pnevmatikakis “Sports Indexing Through Camera and Content Understanding”, The 7th European Conference on Visual Media Production (CVMP 2010), London, UK, 17-18 Nov 2010
  59. A. Conconi, F. Cattaneo, A. Pnevmatikakis, J. Soldatos, S. Prost and M. Tscheligi, “HERMES: A Pervasive System for Memory Support and Ambient Assisted Living”, in Proceedings of the 6th International Conference on Software and Data Technologies, Vol. 1, Seville, Spain, July 2011.
  60. T. Petsatodis, F. Talantzis, C. Boukis, Z.H. Tan, R. Prasad, ”Multi-Sensor Voice Activity Detection based on Multiple Observation Hypothesis Testing”, Proc. of INTERSPEECH, Florence, Italy, Aug. 2011.
  61. M.-D. Albakour, C. Macdonald, I. Ounis, A. Pnevmatikakis and J. Soldatos, “SMART: An open source framework for searching the physical world”, SIGIR 2012 Workshop on Open Source Information Retrieval, Portland, Oregon, USA, Aug. 2012.
  62. T. Petsatodis, F. Talantzis, C. Boukis, “Empirical Mode Decomposition VAD based on Multiple Sensor LRT”, Proc. of IEEE Constantinides International Workshop on Signal Processing (CIWSP 2013), London, UK, Jan. 2013.
  63. A. Pnevmatikakis, A. Stergiou, Th. Petsatodis and N. Katsarakis, “Visual measurement cues for face tracking”, 18th International Conference on Digital Signal Processing (DSP2013), July 2013.
  64. S. Astaras and A. Pnevmatikakis, “Context extraction in the caring home: Infrastructure and algorithms,” IEEE ICC 2015 Workshop on ICT-enabled services and technologies for eHealth and Ambient Assisted Living, London, UK, June 2015.
  65. A. Pnevmatikakis, “Virtual sensing environment for care at home,” Global Wireless Summit, Hyderabad, India, Dec. 2015.
  66. S. Astaras and A. Pnevmatikakis, “Background Subtraction for Activity Patterns in Cities,” First International Workshop on Sensing, Processing and Learning for Intelligent Machines (SPLINE), Aalborg, Denmark, July 2016.
  67. S. Diamantas, S. Astaras and A. Pnevmatikakis, “Depth Estimation in Still Images and Videos Using a Motionless Monocular Camera,” IEEE Int. Conf. on Imaging Systems & Techniques, Chania, Greece, Oct. 2016.
  68. G. Bardas and A. Pnevmatikakis, “Real-time face tracker yielding 3D pose and position,” Global Wireless Summit, Aarhus, Denmark, Nov. 2016.
Head of Multimodal Signal Analytics Group