Tutorials & Workshops

	Important Dates
	
            Time
            Duration
            Title
            Speakers
		
			June 5
			120 min
            Tutorial 01 - Semantic Communication for Multimedia Applications
            Anil Fernando & 
 Yasith Ganearachchi
		
			June 5
			120 min
            Workshop 01 - Wave your way: Deploy Personalized Gesture Recognition using ToF Sensors with MATLAB and STM32Cube.AI Dev Cloud
            Shixin Zhuang & 
 Danilo Pietro Pau

T-01: Semantic Communication for Multimedia Applications

Authors:

Anil Fernando - University of Strathclyde, United Kingdom
Yasith Ganearachchi - University of Strathclyde, United Kingdom

Abstract: Semantic communication was first discussed by Claude E Shannon and Warren Weaver in 1949, when they classified communications as a problem with three levels: physical, semantic, and effectiveness. The physical problem concerns itself with accurate transmission of the data content of a message and led to the birth of information theory, while the semantic problem deals with ensuring the meaning (or semantic) of a message is delivered, and the effectiveness problem is whether the intended action by the message was achieved. Although physical communications evolved at an exponential pace from the early days of information theory, creating the foundation on which the current gaming, entertainment, and media ecosystems are built, semantic communication was not explored further, mainly due to the lack of appropriate tools for its implementation. However, with recent advancements in deep learning and computer performance, developing semantic communication as a useful paradigm to improve the capacity and reliability of communication systems has become a possibility. We explore how the concepts behind semantic communications can be used to complement conventional multimedia communication systems, with special focus on image compression and transmission and video compression and transmission. The early results show promise in achieving better quality reconstructions of images and video for a given bandwidth compared to state-of-the-art image and video compression techniques, but also have several key challenges to be overcome to be commercially adapted. We present the background, current state, and future roadmap for using semantic communications for multimedia applications.

Keywords:

Semantic Communications
Image Compression
Video Compression

Outline:

Introduction (5 minutes);

Concept of Semantic Communication (10 minutes);

AI/ML Techniques for Semantic Communication (10 minutes);

Semantic Extraction in Multimedia Applications (20 minutes);

Semantic Communication Systems for Images (20 minutes);

Semantic Communication Systems for Video (20 minutes);

Current Topics in Semantic Communications (10 minutes);

Open Challenges in Semantic Communications (10 minutes);

Questions + Conclusion (15 minutes);

W-01: Wave your way: Deploy Personalized Gesture Recognition using ToF Sensors with MATLAB and STM32Cube.AI Dev Cloud

Authors:

Shixin Zhuang - The MathWorks, USA
Danilo Pietro Pau - STMicroelectronics, Italy

Abstract: Just as speech and handwriting bear the unique signature of an individual, human gestures carry a distinct personal touch. Crafting a one-size-fits-all classifier for gesture recognition poses significant challenges due to the inherent variability among individuals. Accurately recognizing gestures in different scenarios, such as driving and gaming, under variable carry position of a tiny device, remains challenging due to limited and costly data availability. In this hands-on workshop, we present an end to end approach that combines the power of MATLAB and ST machine learning development environments to address the challenges of data preprocessing, model selection and performance evaluation in a systematic, productive, and efficient workflow. Movement-based signal data captured by time-of-flight (ToF) sensors contain patterns specific to different gestures. Deep learning models are trained to automate feature extraction within a gesture dataset gathered from human volunteers. Participants will explore different architectures, layers, and compression techniques. We then introduce STM32Cube.AI Dev Cloud, a cutting-edge platform for deploying and optimizing deep learning models on microcontrollers (MCU). Participants will export the trained models from MATLAB to STM32Cube.AI Dev Cloud. They will gain insights into evaluating performance benchmarks against the stringent requirements of tiny devices. Throughout the workshop, participants will engage in practical exercises, applying the MATLAB workflow to ToF dataset and deploying it to MCU development boards. By the end, they will possess the skills to navigate the complexities of deep learning model selection and performance evaluation to achieve deployment goals in an unprecedented productive way.

Keywords:

Deep learning
Tiny edge deployment
Bayesian Optimization
Micro-controllers
Productivity

Outline:

Introduction to Gesture Recognition with ToF Sensors (20 minutes);
Data Preprocessing and Model Training in MATLAB (15 minutes);
Model Compression (20 minutes);
Deployment with STM32Cube.AI Dev Cloud (20 minutes);
Performance Evaluation and Benchmarking (20 minutes);
Recap and Evolutions (20 minutes).