Assessing emotional state in outer space

PDEng trainees from Eindhoven University of Technology finished an exciting project for the European Space Agency (ESA).
4TU Delft
4TU Eindhoven
4TU Twente
4TU Wageningen
  • Share

PDEng trainees Software Technology at TU/e, recently finished an exciting project for the European Space Agency (ESA). They developed a system, based on machine learning, which detects the emotional state of an astronaut in space from video and audio fragments. A healthy emotional state is very important since it directly influences the chance of success of space missions. The ESA project was part of the first year of the PDEng ST program.

Context

Deep space missions will pose many new kinds of challenges to astronauts’ health since they will be under extreme conditions. Preserving a healthy emotional state will be one of the main hurdles for astronauts since their emotional state directly influences the chance of success of space missions.  Additionally, assessing an astronaut’s emotional state can be difficult since communication with the earth is often delayed, and in the case of deep space missions, might not even be there. Therefore, this must be done autonomously. While emotion recognition has been traditionally applied only to text data, due to recent advancements in machine learning, deep learning, and computer vision, emotion recognition is also starting to be applied to audio and images/video data. These recent advancements open the possibility to make a facial and voice emotion recognition system, to be deployed on-board of a spacecraft during a space mission, to assess the emotional state of an astronaut in real-time.

Goals

The goal for trainees in this project was to develop a system that detects the emotional state of a person from video and audio fragments using machine learning. These models should be able to be deployed on limited hardware, therefore it was limited to a low cost, credit-card sized computer (Raspberry Pi 4).

In addition to this project goal, the PDEng Software Technology program had another goal as well. They let the trainees explore their potential by giving them certain roles and responsibilities that exist in the industry (e.g. Project Manager, Team Leader, Software Architect).

Contribution to ESA research

The trainees gained and explored a lot of information on deep-learning models. They developed a modular and extensible machine learning framework as well. The models are not accurate enough to be used in a real environment yet, but they contribute a lot to the research currently being done by ESA.

The trainees wrapped-up their project by giving a public presentation, which was live-streamed to the public for the first time in the ST program. This was possible because of the open-source nature of the project. Interested? Check here

Experiences of PDEng trainee Juan van der Heijden

Juan van der Heijden was appointed project manager, responsible for aligning requirements with the stakeholders, translating the expectations of the stakeholders into a project plan, regularly reporting the progress to the stakeholders, and making sure the project was achievable. And last but not least: keeping the team spirit high. “ I have had a lot of fun doing this since it’s a quite non-technical task in a high-tech field. Doing this I was able to train my non-technical skills like communicating, negotiating and presenting”.

Juan did both his bachelor's and masters at the TU/e: bachelor Software Science and masters in Computer Science and Engineering. During his bachelor and master program, he worked as a high school teacher in Informatics. After his masters, he didn’t feel completely ready for the industry yet. Many courses and his thesis were very theoretical and academically focused. Most of his technical experience was in university projects and more fun projects on the side. “I was missing the connection with the industry. That’s when I got introduced to the PDEng Software Technology program, which was exactly what I was looking for. The program trains you for an industrial career as a technological designer. When I heard what kind of exciting projects you get to work on, I was sold.” He enjoyed his first year of the PDEng program a lot and had so many valuable experiences that he missed before. He is looking forward to his individual project next year at Philips research.

Assessing emotional state in outer space

PDEng trainees Software Technology at TU/e, recently finished an exciting project for the European Space Agency (ESA). They developed a system, based on machine learning, which detects the emotional state of an astronaut in space from video and audio fragments. A healthy emotional state is very important since it directly influences the chance of success of space missions. The ESA project was part of the first year of the PDEng ST program.

Context

Deep space missions will pose many new kinds of challenges to astronauts’ health since they will be under extreme conditions. Preserving a healthy emotional state will be one of the main hurdles for astronauts since their emotional state directly influences the chance of success of space missions.  Additionally, assessing an astronaut’s emotional state can be difficult since communication with the earth is often delayed, and in the case of deep space missions, might not even be there. Therefore, this must be done autonomously. While emotion recognition has been traditionally applied only to text data, due to recent advancements in machine learning, deep learning, and computer vision, emotion recognition is also starting to be applied to audio and images/video data. These recent advancements open the possibility to make a facial and voice emotion recognition system, to be deployed on-board of a spacecraft during a space mission, to assess the emotional state of an astronaut in real-time.

Goals

The goal for trainees in this project was to develop a system that detects the emotional state of a person from video and audio fragments using machine learning. These models should be able to be deployed on limited hardware, therefore it was limited to a low cost, credit-card sized computer (Raspberry Pi 4).

In addition to this project goal, the PDEng Software Technology program had another goal as well. They let the trainees explore their potential by giving them certain roles and responsibilities that exist in the industry (e.g. Project Manager, Team Leader, Software Architect).

Contribution to ESA research

The trainees gained and explored a lot of information on deep-learning models. They developed a modular and extensible machine learning framework as well. The models are not accurate enough to be used in a real environment yet, but they contribute a lot to the research currently being done by ESA.

The trainees wrapped-up their project by giving a public presentation, which was live-streamed to the public for the first time in the ST program. This was possible because of the open-source nature of the project. Interested? Check here

Experiences of PDEng trainee Juan van der Heijden

Juan van der Heijden was appointed project manager, responsible for aligning requirements with the stakeholders, translating the expectations of the stakeholders into a project plan, regularly reporting the progress to the stakeholders, and making sure the project was achievable. And last but not least: keeping the team spirit high. “ I have had a lot of fun doing this since it’s a quite non-technical task in a high-tech field. Doing this I was able to train my non-technical skills like communicating, negotiating and presenting”.

Juan did both his bachelor's and masters at the TU/e: bachelor Software Science and masters in Computer Science and Engineering. During his bachelor and master program, he worked as a high school teacher in Informatics. After his masters, he didn’t feel completely ready for the industry yet. Many courses and his thesis were very theoretical and academically focused. Most of his technical experience was in university projects and more fun projects on the side. “I was missing the connection with the industry. That’s when I got introduced to the PDEng Software Technology program, which was exactly what I was looking for. The program trains you for an industrial career as a technological designer. When I heard what kind of exciting projects you get to work on, I was sold.” He enjoyed his first year of the PDEng program a lot and had so many valuable experiences that he missed before. He is looking forward to his individual project next year at Philips research.