Just add AI for expert astronaut ultrasound

“The success of crewed exploration comes down to the health and safety of our astronauts,” explains ESA biomedical engineer Arnaud Runge, overseeing the project. “As missions venture further into space that becomes something that is harder to ensure because the number and skills of crewmembers will be limited. Therefore we need technological assistance to make future crews less and less dependent on Earth-based expertise”

Living in a constrained volume in the sustained absence of gravity while being exposed to high levels of radiation can affect many critical organs, as well as leading to balance disorders, fluid shifts, alterations in visual functioning, cardiovascular deconditioning, decreased immune function, muscle atrophy and bone loss. In addition, future planetary missions might lead to injuries during surface operations.

The good news is that most of these conditions can be monitored using ultrasound imaging, relying on echoes from sound beyond the hearing range of our ears to open windows into the soft tissues of the human body. The bad news is it requires years of training to make someone proficient in performing an ultrasound exam.

“Ultrasound imaging has already become an essential diagnostic tool for International Space Station crews,” comments Carlos Illana of GMV in Spain, the company leading the project consortium for ESA. “But in the current practice on ISS, the astronaut who is applying the ultrasound device to their crewmate is either receiving real-time guidance from an experienced ultrasound operator down on the ground or performs the investigations based on the limited training received prior to the mission.”

Arnaud adds: “To overcome this challenge, ESA has previously worked on the concept of robotised tele-ultrasound, where the expert radiologist on Earth was remotely piloting remotely the ultrasound probe aboard the ISS. However, while interesting for utilisation on the ISS as well as for terrestrial applications, this approach also has limitations: Indeed, once crewed missions extend beyond Earth orbit into deep space such guidance will no longer be feasible, because the greater distance from Earth gives rise to an increased time-lag in communications, while bandwidth will be constrained as well.”

There is therefore a need for solutions providing crew with more autonomy.In response, ESA’s Autonomous uLtrasound Image Improvement SyStEM, ALISSE, offers astronauts the on-the-spot ability to capture diagnostic quality ultrasound images as if they were expert radiologists, thanks to the assistance of AI and machine learning.

Collaborating with the project, the Nuclear Physics Group of the Universidad Complutense Madrid devised new techniques for ultrasound simulation and image synthesis while the Emergency and Urgency Radiology Service of La Paz Hospital in Madrid provided guidance in ultrasound examinations and pathologies, as well as the provision and labelling of hundreds of thousands of anonymised ultrasound scans, used for training the deep learning neural network that underlies the ALISSE system.

Arnaud adds: “La Paz is the biggest hospital in Spain, performing more than half a million ultrasound examinations per year in the Emergency Service alone, using more than 40 different device models. We used an active learning mechanism to filter out non-interesting images, leaving less than 2% that the Radiology Service selected and labelled for our Neural Network Training Subsystem to be trained on.”

This amounts to a huge quantity of curated images of more than 50 000 patients per organ, including plenty of examples of ‘pathological’ – or diseased – cases. For the initial ALISSE protype the consortium explored kidneys and bladders, as very representative abdominal organs which are not easy to scan, related to common astronaut diseases such as stone formation and urinary retention.

David Mirault of GMV says: “As we developed the system, ESA flight surgeons gave us essential feedback and guidance. Our aim was to make the user interface as intuitive as possible, so we had a group of entirely untrained physics students try out using it. Ultrasound images are noisy, blurry and contain plenty of artifacts such as shadows and speckle noise, and everyone’s body is different. So medical professionals need years of specific courses and training to learn this diagnostic technique for a single organ. This means the chances of an untrained novice performing a successful ultrasound exam are essentially zero.”

However ALISSE users receive detailed guidance of where in the body to place the ultrasound wand, are provided with example images of the target organ and given the percentage likelihood of the object in view being the correct target. The system is also able to differentiate between the clinically valuable long ways ‘plane detection mode’ for an organ versus a less useful ‘transverse’ side view.

Jon Scott, supporting the project at the European Astronaut Centre, comments: ‘The end results are very encouraging; nine out of ten of the ALISSE-assisted students’ images were clinically acceptable ultrasound standard planes of kidneys and bladders, approaching the performance of a trained radiologist. And as an added benefit, ALISSE can also work with multiple ultrasound devices, maximising its flexibility and reducing the barriers to its implementation. The result is a system that allows astronauts to take more responsibility for their own medical care, an essential feature for the future of space medicine, and should also democratise the use of ultrasound imaging back on Earth. With the continued development of this technology, we can look forward to a time when frontline medical partitioners can employ AI-guided ultrasound devices as proficiently as they collect blood samples today.”

The ALISSE project was supported through ESA’s Technology Development Element, fostering promising new technologies for space. As a next step, the consortium plans to increase the system’s support to other organs and improve guidance instructions to make ALISSE even more intuitive. ESA is also interested in having the ALISSE system working on a tablet connected to an ultrasound probe.