It’s midday on Mars. The scarlet skies are clear and it’s a balmy zero degrees – the perfect conditions for a day out.
One by one, the sightseers hop off the bus and sink their feet into the dust. The desolate, rocky landscape stretches to the horizon. Above them, they can just discern the faint outline of the planet’s larger moon, Phobos. They take some selfies.
There are no spacesuits, tour guides, or humans in sight. This is a party of robot avatars – and today they’re going to climb Olympus Mons, the largest volcano in our Solar System.
The first image of Mars was taken by the Viking 1 lander after it touched down in 1976 (Credit: Nasa)
Over 33.9 million miles (54.6 million kilometres) away on Earth, the real tourists are enjoying a spot of inter-planetary exploration from the safety of their own living rooms. They’re hooked up to sensor-filled suits which allow their body movements to be directly mirrored by their avatar, in real-time. Crucially, the tourists can see, hear and feel everything their avatar can, from the contours of the rocks underfoot to the weight of their cameras in their hands.
If you think this sounds far-fetched, think again. Though we’ve dreamed of setting foot on the Red Planet for centuries, with 20-month return flights, an average temperature of -55C and a price tag of $10bn per ticket, it’s hardly the ideal holiday destination.
Enter robot avatars, intrepid explorers to boldly go where humans can’t
That’s not to mention the risky rocket launch – the chances of exploding are around 1 in 100 – or the possibility of crash-landing at 12,000mph (19,200km/h). If you make it, you’ll be rewarded with a pelting of cosmic radiation equivalent to 10,000 CT scans and a gravity situation (62% less than on Earth) which could turn your bones to matchsticks or make you blind. Finally, there’s the small problem that the first trip to Mars isn’t planned until 2024. And it’s one-way.
Enter robot avatars, intrepid explorers to boldly go where humans can’t. By combining sophisticated sensors with the latest virtual reality technology, robots can paint a three-dimensional picture of the world around them and convey it back to a human controller. The technique promises to recreate an experience as vivid as if you were admiring the Red Planet through a spacesuit. And incredibly, the technology to make this happen already exists.
For a start, there are already robot avatars on Mars. “We’re kind of doing it already,” says John Callas from Nasa’s Jet Propulsion Laboratory. The exploration rover Opportunity – sister to Spirit, which lost signal in 2012 – has been providing an immersive, three-dimensional perspective of the Martian surface ever since it landed in 2006.
540,000 foreign tourists dive the Red Sea’s coral reefs every year (Credit: Ryckaert Marc/ Wikimedia Commons)
The innovation which makes this virtual exploration possible is surprisingly simple. The rover has “stereoscopic vision” – the fancy term for having two eyes facing in the same direction, like a human’s. By comparing tiny differences between its two fields of view, the rover is able to calculate depth, an object’s size and how far away it is.
“You can stand in a hollow room, if you will, and feel as though you are looking around on Mars and walking around in that scene,” says Callas, who is project-managing the expedition.
Back in 2012, a team of Stanford University scientists took this approach a couple of giant, low-gravity leaps further. Led by Oussama Khatib, the team was looking for a way to explore another alien environment: under the sea. The Red Sea is home to one of the most biodiverse coral reefs on the planet, with over a thousand species of fish living alongside rare dugongs, sea turtles and sharks.
Together with colleagues from Stanford University, Khatib developed Ocean One, a humanoid robot like no other
But it’s under threat. In the Egyptian city of Hurghada, which is near 28 miles (40km) of reef, there were only 565 hotel rooms in 1989; by 2006 there were more than 48,000. That’s 84 times more tourists in less than two decades. The challenge is to find out what lives there – and lobby to protect it – before it’s too late.
And therein lies the second problem. Diving is time-consuming, dangerous and generally restricted to shallow depths, while robotic submersibles are notoriously clumsy. What scientists needed was the abilities of a human diver without our mortal weaknesses. (To see what the robot looks like, check the video below)
Together with colleagues from Stanford University, Khatib developed Ocean One, a humanoid robot like no other. Resembling a cross between C-3PO and a crash test dummy, this masterpiece of engineering comes with tactile abilities to rival a normal human.
Each hand is fitted with force sensors which relay information back to a controller in real-time, allowing them to feel what their avatar feels and control how it moves around. The result is a robot with super-human resilience and actual human brainpower. It’s as though the researchers are underwater, without having to even get their toes wet.
Even simple tasks like walking can be a challenge for robots (Credit: Master Chief Petty Officer John Williams/Wikimedia Commons)
At the moment, Ocean One is controlled using joysticks. But others have developed robots which work more like hi-tech puppets, where the robot directly mirrors the actions of the controller. When the human moves their arm to the left, the avatar moves its arm in unison. All the system needs is for the human to don a sensor-filled suit.
Earlier this year, Khatib used his robot to retrieve a delicate glass vase from the ancient wreck of La Lune. The flagship of Louis XIV of France sank off the coast of the city of Toulon in 1664.
To get to grips with why this technology is so revolutionary, first we need to understand the limitations of a regular, autonomous robot.
When it comes to simple, everyday tasks – such as moving around a room without bumping into anything – robot IQs are decidedly underwhelming
While they can crunch datasets which would take puny humans millennia in milliseconds, when it comes to simple, everyday tasks – such as moving around a room without bumping into anything – robot IQs are decidedly underwhelming.
If you’ve ever watched a robot move around in an unfamiliar environment, you’ll notice that they spend a frustratingly large proportion of their time completely still. They’re “motion planning” – trying to figure out the most efficient path between two objects, such as rocks. “These tasks have very nasty mathematical properties,” says Jonas Buchli, an expert in agile robotics at the Swiss Federal Institute of Technology.
The red dust in the Martian atmosphere actually means sunsets are blue (Credit: Nasa)
Then there’s the problem of gripping. It’s surprisingly difficult to design a robot to use the right amount of force without going over the top, since it takes experience to know how much force different materials and objects can take.
If you instructed most autonomous robots to pluck a vase from an ancient wreck, they’d probably smash it to pieces (or drop it) and present you with the debris. “A human remotely operating a robot arm can perform more manipulation tasks than what’s currently achievable with purely autonomous robots,” says Pratt.
Even with artificial intelligence, robots still need at least some guidance from a human. Take self-driving cars. As of August this year, Google’s cars had travelled 170,000 miles. That’s equivalent to nearly seven times around the globe. Surely if cars can travel around the world on their own, we could use similar technology to navigate on Mars?
Robots lack the life experience to make good decisions and are easily confused
“One has to realise that this is all on the back of huge datasets which are collected ahead of time,” says Buchli. In fact, the cars aren’t forging new routes – they will have already been on those same roads several times, directed by a human driver.
In situations they’ve never encountered before – so-called “unstructured” environments – robots lack the life experience to make good decisions and are easily confused. “If an astronaut is being sent to another planet he is probably 40 years old right? So he’s collected a lot of information about how the world works,” says Buchli.
Buchli gives the example of crevices or gaps in the Martian landscape. While humans would immediately recognise these features, many robots would struggle. If the lighting conditions weren’t right – brighter or darker than the robot was used to – you can forget it.
Nasa has had a permanent colony of robot avatars on Mars since 2003 (Credit: Alamy)
“Avatars are a way to side-step some of these problems because people are very good at solving these complex problems and can learn to operate complex machinery,” says Buchli.
Khatib’s team is currently experimenting with the possibility of mind-controlled avatars. In 2012, as a result of research by a different team, a person lying in an fMRI machine was able to control a robot just by thinking about moving.
Eventually, Khatib hopes his robots will be roaming other planets; Nasa is already developing its own humanoid avatars, in the hope of using them to explore Mars. “Ideally, they would be so good that a human could get to the same places and do the same work that they could if they were actually there,” says Jerry Pratt, a robotics expert from the Institute for Human & Machine Cognition who is working on the Nasa project.
Basically, the roundtrip delay of a signal to Mars is around 15 minutes – Jonas Buchli, Swiss Federal Institute of Technology
But we’re not quite there yet. There’s one final hurdle to cross before virtual, inter-planetary tourism is possible – and unfortunately, it’s the laws of physics. At the moment, Nasa communicates with its rovers on Mars using radio waves, which travel at the speed of light (around 300,000 km a second). That’s very, very fast – but Mars is very, very far away. “Basically, the roundtrip delay of a signal to Mars is around 15 minutes,” says Buchli.
This is a big problem. So far, Opportunity has travelled more than 26 miles (43 kilometres) over the surface of Mars. But it’s taken 12 years. Because it has to wait for instructions from back on Earth, in a given day, it can’t drive more than 100 metres.
Pratt is optimistic. He envisages a robot colony of rovers and humanoids on the Red Planet, with some relying on artificial intelligence and others acting as avatars for humans elsewhere. “These humans might be in a habitat on the surface, or in orbit around Mars, or on one of the moons of Mars,” he says.
Who knows, perhaps one day the first humans to colonise Mars will send avatars back for sightseeing on Earth, too.
If you liked this story, sign up for the weekly bbc.com features newsletter, called “If You Only Read 6 Things This Week”. A handpicked selection of stories from BBC Future, Earth, Culture, Capital, Travel and Autos, delivered to your inbox every Friday.