Space exploration is one of humankind’s greatest undertakings, prompted by a curious mind and a trust for knowledge of what lies beyond. The latest monumental example of our human curiosity and technological advancements is the safe landing of the NASA rover Perseverance last Thursday, which marked the beginning of the ambitious search for past life on Mars. Perseverance’s task is to gather soil and rock samples to be studied afterwards for a potential sign of life. Apart from collecting rock samples, the rover also provided us with new images of the Red Planet and the first-ever recorded video and audio of Mars.
The Perseverance rover is not alone on Mars. It’s accompanied by the first helicopter sent to another planet – Ingenuity. The 4 pounds (1.8 kilograms) helicopter in the size of a tissue box marks several other colossal steps in space exploration. Besides being the first aerial flight on another planet, Ingenuity is the first aircraft to attempt controlled flight on another planet, which is extremely difficult considering Mars’ thin atmosphere and bone-chilling conditions. The helicopter is an innovative experiment, but if it succeeds, it will enable a new aerial dimension of future Mars exploration with other advanced robotic flying vehicles that could provide access to terrain that is difficult for rovers to reach or too distant for orbiters high above. Both Perseverance and Ingenuity use AI throughout their mission to successfully land, enable engineers to remotely aim and control the rover’s chemistry camera and navigate autonomously.
All previous or current NASA’s missions in space exploration have been enabled by AI. We’ve previously touched upon how NASA uses AI and machine learning for space exploration in the teaser article for Shreyansh Daftry’s Data Innovation Summit session. In order to appreciate the great success of NASA and humanity, we are again exploring the current topic, but with a more in-depth perspective. As an AI Research Scientist at NASA Jet Propulsion Laboratory, Shreyansh introduced us to the evolution of autonomous robots for space exploration with AI and ML, challenges and solutions to training deep learning models in the Martian environment, and lessons and future AI-powered exploration.
Why space exploration needs AI
NASA has been building capable spacecraft that have visited almost all the major object in our Solar System, and exploring new frontiers in interstellar space for the past 60 years.Â
However, space exploration in the past didn’t always include intelligent machines. In fact, the Apollo mission that carried the first humans to the moon performed a proper landing thanks to Neil Armstrong’s abilities, who manually piloted the spacecraft. Another example is the Curiosity rover, which is one of the most capable space machines, but it’s remotely operated by humans from the mission control room on Earth.
Although man-operated space machines worked in the past, they are not scalable for future missions. “There is a growing need for future spacecraft to be autonomous, self-aware and be able to make critical decisions on their own,” explained Shreyansh. There are two main factors for this. The first factor is the difficulty with deep space communication and critical missions, especially involving the outer planets of our solar system. Due to the large distance and the limited bandwidth of radio signals, the only way the far-off planets can be explored is by sending fully-autonomous robots. The second factor is scalability. Future missions will require many more satellites and space assets that would require 100+ engineers and scientists controlling them remotely from the Earth. If we want to scale out the space economy, our space assets need to be self-sustainable. AI is a crucial ingredient in making both factors a reality.
Shreyansh outlined several key capabilities that would require AI-powered space exploration for future robotic missions, such as autonomous navigation, autonomous science and mission operations, scheduling and planning, precision landing, autonomous manipulation, allowing robots to perform dexterous operations and future missions with human-robot teams. There are numerous other AI applications in space exploration beyond robotics and automation, but they are outside of Shreyansh scope of work. In his talk, he focused on AI application in autonomous navigation.
AI in autonomous navigation at NASA
To navigate rovers safely on the Mars surface, NASA uses surface or orbital images. Human operators use geometric and textural information to assess terrain safety.
In contrast, different kinds of NASA navigation software uses either only geometric information or topographical information, which doesn’t provide full terrain textural information. This has resulted in the rover getting stuck in send on previous Mars missions.
NASA created SPOC – a deep learning-based terrain classifier for Mars rovers, Shreyansh explained, to solve this obstacle in navigation. SPOC can tell the terrain types on images based on textural features. They have developed two versions of the SPOC algorithm – one that uses rover images (left) and one that uses orbital images (right), as seen on the screenshot below.
With SPOC, NASA has attempted to enable a human-like ability for algorithms to assess the traversability and safety of rover driving based on the topography and the terrain type.
Challenges and solutions to training deep learning models on Martian environment
Thanks to the ongoing developments in computer vision, deep learning and AI, NASA has made great strides in their space exploration missions. However, adopting the deep learning models and neural networks to Martian conditions presents an exponential challenge, or rather a set of challenges, stated Shreyansh, such as availability of data, severely limited onboard computation and system validation to gain trust.
To secure data sets for training and transfer learning, they asked their rover drivers and scientists on the Mars rover mission to label a small amount of data with high-quality labels. To augment the labelled data, they launched the AI4Mars citizen science project using free crowdsourcing from the internet community to label the Curiosity images using a web-based tool. To overcome the additional challenge of people’s lack of knowledge and the noise in the labelled data, they developed a workshop-style tutorial and interactive tool in 10+ languages.
Additionally, they looked into the rapidly developing area of high-fidelity simulation and generating synthetic data for improving their deep learning pipelines. Based on this method, they successfully created the SPOC classifier to segment Mars terrain types.
To tackle the limited computation challenge, they deployed the SPOC-lite algorithm on a test rover and tested. SPOC-lite can easily identify sandy surfaces from rock surfaces.
The third challenge of system integration and end-to-end verification is the most overlooked in the data science and machine learning community, but it is especially important in highly-complex safety-critical cyber-physical systems like spacecraft, highlighted Shreyansh. Deep learning algorithms and neural network models are instrumental in training the rovers, but they are inherently “black box”, presenting a major obstacle in model validation and performance. When the rover is launched in space, they can’t fix things if something goes wrong. As their machine learning system deployment has to work on the first shot, it requires exhaustive system integration, validation and testing campaign, which involves creating scenarios that can be expected on Mars and going crazy with testing.
Future AI-powered NASA missions
Sending Perseverance and Ingenuity to Mars was a colossal achievement for NASA and AI. In future, NASA plans to send similar rovers to the Moon, Mars and beyond. The next few decades of robotic space exploration will involve the outer planets and their moons, stated Shreyansh.
To prepare for the planned mission on Jupiter’s moon Europa, NASA has been developing robots that can learn how to walk, ski, drive on its icy surface and dive in the oceans to look for evidence of life. The robots have to be completely autonomous and without any human involvement for several hours on end. An additional challenge is the unknown subterranean nature of the environment. “This is one of the greatest challenges for AI and robotics for the century,” adds Shreyansh.
As for future Mars missions, the next ambition is to explore the caves and lava tubes that have the highest potential for a human colony. To build capabilities for navigating the steep slopes, NASA has been developing the LEMUR robot capable of climbing almost vertical slopes.
Looking forward, one of the most exciting NASA undertakings would be AI-powered human exploration, where humans and robots would go on exploratory missions together. So the next milestone for Shreyansh and his team would be exploring how they can enable human Mars exploration and the role AI would play in it, especially in the context of human-robot teams.
Add comment