Henrik Matzke is in the driving seat of a car, poised to make a very unusual manoeuvre. The car pulls up to a junction. He concentrates for a moment, willing the car to turn. The steering wheel spins, and the car veers to the right, accelerating away.
With his hands on his lap, Matzke is driving the car with thought alone, often at speeds up to 50km/h (31mph).
He’s part of a team at the Free University of Berlin working on what they call the Brain Driver – a project that’s hoping to bring research into reading and interpreting brain signals into people’s cars and homes. What is it like to control a one-and-a-half-tonne vehicle with your mind?
The original premise behind Brain Driver was to build a system that someone with a physical disability can use to manoeuvre through the world – as well as cars, the team has developed the technology for wheelchairs too. Brain Driver would, in theory, allow them to drive by simply thinking “right” and “left” and “forward".
But turning that dream into a reality is as hard as it sounds. Adalberto Llarena, a roboticist with the Brain Driver project, says the team has faced two main challenges: the hardware and the humans. On the hardware side, they’re trying to design a commercially viable piece of equipment that can listen in on the brain’s whispers and turn them into meaningful signals that power a machine. On the human side, they’ve got to develop something that real people can actually learn to use.
A member of the Brain Driver team wears the headset used to control their car (Brain Driver project)
The Brain Driver consists of a headset with 16 sensors that monitor electrical signals from the brain. Clinical devices usually use 32 sensors, but Llarena and his team are trying to build something that’s as small, cheap and unobtrusive as possible. “We think that 16 are probably too many,” he says. “We’re trying to figure out if we can take half of them out.” But there’s only so much streamlining they can do before the signals get too weak. They’re already working with tiny, microvolt signals from inside the brain and trying to read and interpret them. “It’s as if we were putting a small microphone on one side and trying to hear one of a million people shouting on the other side,” says Llarena.
After reading these signals through the skull, the system has to turn them into instructions for the wheelchair or car. The idea is that the driver thinks one distinct thought to turn right, and another left, while the electrodes pick up the associated activity. But those thoughts won’t necessarily be as simple as the words “left”and “right”; it could be something more abstract, like a certain place or a shape. “It was a long process because I didn’t know what to do at all. I was thinking of everything, the beach, red cubes, red circles.” Eventually he figured out that if he pictured a red cube in his mind, and then imagined that cube moving forward in his skull he could make the machine move forward. If he thought about that cube moving left, he could go left. Later on, the team realised that what he was really doing was activating his motor cortex, a process that sent out a strong enough signal to detect.
Henrik Matzke drives with thought by inducing an orange cube to move across a screen (bottom right); the car then calculates a trajectory (top right) (Brain Driver project)
“In my case it was quite easy,” says Matzke, which prompts the rest of the team to laugh. “The other guys are laughing because it’s not working for them,” he explains. And that’s the human challenge in making something like this work – training your brain to produce signals that the machine can interpret is really hard. Even Matzke, who was the natural in the group, said it took months to get confident enough to actually use a car or wheelchair. “I got confident to about 70%,” he says “but you can’t get into a car and say, 'I’m 70% confident'.” After months of training, he was able to manoeuvre a car through a course on a former airport – where there would be no risk of collision should a stray thought pop into his head (the car is not approved for public roads when under mind-control).
He’s nonchalant about the experience. “It’s not so weird,” he says, “because we’ve already developed autonomous cars. If you’re sitting in a car that’s already driving itself, it’s not that weird to drive it with your brain,” he says.
But there are limitations. Right now, the instructions are binary – there’s no way to make a slight left, or a slight right. Nor is there a way to control the speed moving forward. You also need to maintain total focus and relaxation while driving.
The thought-control tests were run at a former airport, to ensure safety (Brain Driver)
Such difficulties have already frustrated owners of thought-controlled prosthetic arms. Training to use these requires months of work – a process many amputees find tiresome. Some amputees abandon the arm, saying it’s just too hard to use. Llarena wants to avoid the same thing happening with brain driving, so they’re working on simplifying the system. It’ll rely less on turn-by-turn instructions from the brain, and more on allowing the brain to select locations and letting the chair or car do the rest. So, rather than steering the wheelchair around each individual turn, the user could simply think the signal for “kitchen” and the chair would take them there.
In the future, implanting electrodes in the brain could allow much finer control, says Omar Mendoza, an expert in brain signal processing who works with Llarena. “You can get really good results in those cases,” he says – although even severely disabled people might be reluctant to have brain surgery to restore their mobility.
Sam Schmidt controlled this racing car using only head movements (USAF/Al Bright)
Llarena and his team aren’t the only ones trying to develop cars and wheelchairs for people who can’t physically power them. A few years ago, Toyota worked on a brain-controlled wheelchair that users could start, stop and turn with their minds. And one team recently got a quadriplegic former race car driver behind the wheel again. Rather than using brain signals, he controlled the car by tilting his head and gnashing his teeth.
So far, nobody is ready to release brain-powered cars onto the roads or wheelchairs into the home. Before that can happen, they need an easier system to use and more robust algorithms that can jump in when a driver gets distracted or confused. “On one side we have the programs, and on the other side we have the people who need to use these tools,” Llarena say. “But in the middle we have the problem.” That is the gap that future designs will need to bridge, certainly before you see a thought-controlled car overtaking you on the road. For now Matzke is one of the few people in the world who has hit the road with his mind alone.
Have you had a futuristic experience? Or know someone who has? Get in touch at firstname.lastname@example.org.