The directions are simple, the conditions less so: press the corresponding physical button indicated on the headset display while experiencing weightlessness. Participants in an experiment running on ESA’s 66th parabolic flight campaign are helping researchers to develop augmented reality as a useful tool for astronauts on the International Space Station.
Detailed instructions displayed on a laptop often require astronauts to interrupt their workflow and concentration to refer back to checklists. By replacing static displays like laptops with augmented reality headsets, a team at the University of Rostock aim to increase astronauts’ efficiency and accuracy when working science experiments on the Station.
To begin creating a useful device, developers need to understand how users interact with their environment. In the case of space, the lack of a perceivable up or down makes for a unique frame of reference that affects hand–eye coordination and similar skills. Researchers have boarded an aircraft to work with augmented reality systems in the few seconds of weightlessness these specialised flights provide.
The participant pictured is wearing an augmented reality headset displaying 12 targets arranged in a circle as well as motion-capture sensors to track body movement. During weightlessness, she is shown a target that must be touched on a fixed, vertical board in front of her. Developers track her performance to understand field of view and visual-motor skills under weightless conditions. This feedback is then used to tailor the augmented reality software to the subject’s performance in this unique environment.
Essentially, the augmented reality headsets aim to replace the laptop displays that the astronauts currently consult for instructions during science operations on the Station. Developers expect to make technical changes based on the performance measures and to run the experiment in future parabolic flight campaigns, with the ultimate goal of getting the hardware ready for testing on future astronaut missions on the Station.
The 66th parabolic flight campaign is being run by Novespace in Bordeaux with sponsorship from ESA, the DLR German Aerospace Centre and France’s CNES space agency.