DFG Research Unit 5336 “Learning To Sense”
The Confluence of Machine Learning and Sensor System Development
The German Research Foundation (DFG) has selected L2S as one out of eight research units in Germany that conduct dedicated fundamental research on artificial intelligence along with an interdisciplinary partner field, in this case sensor system development. The project is situated at ZESS where it can build upon more than 30 years of experience in the field of fundamental and application-oriented, interdisciplinary, research.
Sensor System Development
Developing and optimising the next generation of CMOS Sensors, THz imaging systems, and 3d microscopes taylored to specific automatic data analysis applications.
Sensor System Simulation
Developing faithful simulators of the three sensor modalities in order to know how the recorded data changes as the design parameters of a sensor system are changed.
Machine Learning
Developing new approaches to jointly optimize for the sensor system design along with neural networks parameters.
![The PIs of the DFG-Research Unit "Learning to Sense" at ZESS. Front, ltr: Prof Ivo Ihrke, Prof Bhaskar Choubey, Spokesperson Prof Michael Möller, rear, ltr: Prof Volker Blanz, Prof Andreas Kolb, Prof Margret Keuper. Not pictured: Prof Peter Haring Bolívar [Photo: Sascha Hüttenhain]](http://www.zess.uni-siegen.de/wp-content/uploads/2024/10/DFG-Research-Unit-Learning-to-Sense-at-ZESS_Copyright-Sascha-Huettenhain-1024x729.jpg)
Front, ltr: Prof Ivo Ihrke, Prof Bhaskar Choubey, Spokesperson Prof Michael Möller, rear, ltr: Prof Volker Blanz, Prof Andreas Kolb, Prof Margret Keuper. Not pictured: Prof Peter Haring Bolívar [Photo: Sascha Hüttenhain]
![DFG Research Unit Learning to Sense in Mannheim, March 2024 [Foto: Jan Soehlke/ZESS, University of Siegen]](http://www.zess.uni-siegen.de/wp-content/uploads/2024/10/DFG-Research-Unit-Learning-to-Sense-in-Mannheim_Copyright_Jan-Soehlke_ZESS-University-of-Siegen-1-1024x656.jpg)
For a period of four years (with a possible extension by another four years) a team of seven chairs from Electrical Engineering and Computer Science will closely collaborate on the question how to jointly develop and optimize image sensor system hardware and machine learning approaches to reach optimal performances for specific applications. Our research unit focusses on the development of novel CMOS sensors for visible light, optimal 3d microscopic setups, and optimal sub-surface THz imaging technology along with dedicated machine learning approaches in an application-specific setting.
You can learn more about the project at its website https://www.learning2sense.de/
DFG Research Grant “WASEDO”
Wearable-federated, weakly-supervised Activity Sensing through Egocentric Detection of Objects
Body-worn sensor systems bare a great potential in analyzing our daily activities with minimal intrusion yielding various applications, ranging from the provision of medical support to supporting complex work processes. With (deep) neural networks representing the state-of-the-art technology for the automatic analysis of such data, a key bottleneck becomes the annotation of data for the underlying training, specifically because the acquired non-visual data is difficult to interpret in hindsight, and visual data represents a significant intrusion in the privacy of any participant in such a study. Therefore, the goal of this proposal is to study a federated learning approach, in which clients use both wrist-worn sensors and camera glasses, where the latter deliver visual data that is merely used to locally supervise the training of a network analyzing the wrist-worn sensor signals. By following this weakly-supervised federated learning approach, we are able to avoid both the necessity for manual annotations, as well as the submission of visual data to a central server. Our goal is to conduct fundamental research on the weak supervision as well as the collaborative training to combine our results in a practical solution for wrist-worn sensor data analysis.