“Alexa, turn on the light”; Lockheed Martin, Amazon, Cisco are testing virtual assistants, video conferencing in space | Wbactive

Having flown through its mission with flying colors so far, the Orion spacecraft will soon enter its distant lunar retrograde orbit, with significant testing of various capsule and service module systems progressing.

Inside Orion, teams from Lockheed Martin, Amazon, and Cisco were busy working through the Callisto demonstration, alongside countless experiments and technology demonstrations. Callisto – a combination of Alexa and WebEx video conferencing software – aims to enable virtual assistant presence and reliable video conferencing on future human spaceflights.

In an interview with NASASpaceflight, Rob Chambers – Director of Strategy and Business Development for Commercial and Civil Space, Lockheed Martin – said: “Humanly I’m on the other side of the moon, I’m physically distant, I’m psychologically distant. My bandwidth is limited. I have time lags that only get longer when I go to Mars.”

“How can we increase awareness and interactivity, make humans more efficient and stop wasting brain cells on the simple things computers can do?”

On Earth, one answer to these questions is the virtual assistants built into our intelligent technology. So how about integrating with Orion and future space exploration missions?

That’s what Callisto is designed to test… and a whole lot more.

The Alexa participating in the Callisto demonstration is unable to communicate with critical spacecraft systems such as life support or propulsion. “Alexa can’t cancel the mission or start an engine,” Chambers noted. “And rightly so.”

What Alexa is being tested for on Artemis I falls into the categories of controlling lights in the Orion capsule (responsive to voice commands), properly accessing spacecraft details to respond to queries with accurate information, and as part of a virtual presence device for the Video conference/whiteboard part of the demonstration.

In total, there are more than a thousand utterances created by teams to train Alexa to understand where to look for information in Orion for local information requests.

“If you’re on the spaceship, Alexa needs to know to go to Orion speed and telemetry stream,” Chambers noted.

A bigger part of this is that Alexa needs to be able to understand that different phrases can be used to ask for the same information.

Example: “How fast am I?” and “How fast am I driving?” There are two different ways to query the same data. But when an astronaut on Orion asks one of these questions, they’re actually asking, “How fast is Orion flying?” Alexa needs to be able to understand the meaning behind the question and then know which of Orion’s telemetry streams they’re accessing must in order to find the desired content.

All of this is easier said than done, especially in a spaceship – which isn’t the most acoustically friendly environment for a virtual assistant like Alexa.

This element is tested daily through the use of several different individuals verbally communicating with the Alexa in Orion from Mission Control during times of Deep Space Network (DSN) connectivity and when Mission bandwidth permits.

This also includes ensuring that Alexa can hear properly and respond correctly to different voice patterns and accents in the acoustically imperfect Orion capsule.

Another element to consider is what happens if the requested information can only be gathered by connecting to the internet through the DSN. And what if Orion is out of sight of the DSN at that point?

Another testing area of ​​Callisto includes what Alexa will say during times of communication outages between Orion and Earth, including specifying the time when the outage will end and the request can be answered.

Alexa’s ability to connect to the internet from lunar orbit will also be put to the test, as will lag time given the few seconds it takes for signals to travel the distance between Earth and Orion.

A closeup of the Callisto demonstration interface on Orion. (Source: Lockheed Martin, Cisco, Amazon)

“If we were on the other side of the moon, it would be a few seconds there and back. It’s no big deal. But then you have the switch, you have all the interfaces and handoffs in the systems,” Chambers noted.

“So once I’ve asked a question, it has to come back to Earth to be processed, pull the information from the cloud, send it back to Alexa, and then have Alexa articulate the results. It might be like a total of 10 seconds round trip.”

This delay is governed by the laws of physics, but understanding how human users respond to it can train astronauts for future missions.

The response of human behavior to this delay also provides UI designers with valuable information about what additional behaviors the virtual assistant can perform, e.g. B. playing music or beeping while processing information during the delay so the user understands what they are doing.

But another crucial element of the Callisto demonstration with Alexa is video conferencing. And in order to achieve this with the bandwidth available, Cisco required “state-of-the-art image compression” for the WebEx system.

“We’re trying to do some modern video conferencing at dial-up speeds. It’s not quite as slow, however [we’re] you’re talking about tens of kilobytes or hundreds of kilobytes,” Chambers said.

A team (left) at the Operations Control Center works with the whiteboard portion of the Callisto demonstration during the Artemis I mission. (Image credit: Lockheed Martin)

This test requires Alexa itself to act as a virtual crew member via Mission Control inputs, which are separate from the comparatively simple “Talk to Alexa” tests.

That’s where the off-the-shelf iPad comes in, where Mission Control can test video feeds, image compression, choppyness, time lag, and video-over-voice priority accuracy by simulating a virtual crew member via Alexa and the iPad.

Video conferencing testing is scheduled to last two hours on days the team has access to the Deep Space Network’s 70-meter dishes. These antennas provide the widest possible bandwidth for the tests while Orion travels the furthest distance a mannable spacecraft has ever traveled from Earth.

“So that’s how it works in Mission Control, where we have the Operations Control Center, you’re sitting in front of what we call a desktop pro. This is the Cisco setup. And that has a video camera,” Chambers noted.

“And then we have the speaker and Alexa on board. And underneath is the iPad mounted. So when you speak in mission control, you see Alexa and the iPad on the spaceship, and when you speak to Alexa or she speaks back, the blue ring lights up and your face is of course on the iPad.”

The fourth major test for Callisto also relates to video conferencing and bitrate compression in the form of interactive whiteboards between the crew in Orion and the controllers in Houston.

“We say [you’re in Orion and] They upload an image of the moon for the landing pad,” Chambers says. “You pull up the picture. With [the crew’s] light pen, [the crew] Circle where they will land. Here on Earth you kind of erase that and then highlight it and zoom in.”

“So it’s an interactive whiteboard feature. Now it has this time delay; We can’t fix physics. But in terms of interacting and collaborating and talking about ‘following that particular trajectory’, a picture is worth a thousand words.”

Chambers continued, “So we can test that and confirm that there are no glitches, that it’s running smoothly. It tests everything over the digital network. The algorithms and protocols do the trick.”

Overall, the Callisto demonstration aims to demonstrate and collect data on how common forms of communication in space exploration can be carried forward, not only to extend to space the same comfort and familiarity here on Earth, but also for vital ones , mission-critical communications.

Additionally, the demonstration’s video conferencing element could also be applied on Earth, where government, media and defense operations could all benefit from compressed bit rates while maintaining image quality.

(Lead image: The Callisto demonstration in front of the “Commander” mannequin in the Orion spacecraft during Artemis I. Photo credit: NASA)

Leave a Comment