On 21 March 1919, a committee including a paranormal investigator, a viscountess, a mind reader, a Scotland Yard detective, and a coroner were all assembled in a small flat in Bloomsbury, London. "I have spent years performing with fake mediums all over the world in order to disprove spiritualism," declared their host. "Now at last, I have come across a genuine medium."
The woman who entered the room was wearing a veil that concealed the lower half of her face. She began with a séance which involved a demonstration of "clairvoyance". Each member of the committee had been instructed to bring with them a small personal item or written letter. Before the medium arrived all the objects were placed into a bag, which was then locked inside a box.
The medium held the locked box in her lap, and while the committee watched carefully, she proceeded to not only name the objects within, but to describe them in vivid detail. She divined that one of the objects was a ring belonging to the deceased son of the paranormal investigator, and even read the faded inscription.
Next came a "materialisation" of a spirit. The committee members tied the medium to her chair and the lights in the room were dimmed. The medium appeared to enter into a trance, and a "luminous mist" materialised behind her. One of the committee members later asserted that the mist formed into the shape of an old woman. The form drifted about the room, appearing to pass directly through the medium, before evaporating into the opposite wall.
Was it a genuine glimpse of a world beyond our own? The committee was divided, and while you may not be familiar with most of its members, you have almost certainly heard of the paranormal investigator – Sir Arthur Conan Doyle.
The creator of Sherlock Holmes declared that he was highly impressed with the clairvoyant demonstration, although he said he would need to see the ghost again before he would attest to its paranormality.
Today, Conan Doyle is best known for his detective stories, but the good doctor was also an illustrious paranormal investigator who often failed to see the frauds in front of his eyes. He famously fell for the photographs of the Cottingley Fairies, for instance, faked by two children – Frances Griffiths and Elsie Wright. He attended séances, too. As a spiritualist, Conan Doyle also asserted that he witnessed mediums make direct contact with the spirits of the dead.
The case of the "Masked Medium" was ultimately revealed to be hoax. As an experimental psychologist and magician, however, I believe it is one of two hoaxes that can offer special insight into Conan Doyle's apparent credulity and also some fascinating cognitive illusions that may sometimes affect us all.
You might also enjoy:
• What Sherlock Holmes taught us about the mind
• The man who thinks like Sherlock Holmes
• Should you trust your gut feelings?
Each case involved the activities of magicians. Unlike the spiritualist mediums, they saw themselves as "honest deceivers" who performed illusions for entertainment, and their audiences were fully expected to understand the fact that they were witnessing carefully constructed tricks.
Many magicians – including one of Conan Doyle's own friends, Harry Houdini – were staunch sceptics of the paranormal. Some even took it upon themselves to actively expose fraudulent mediums, both as a public service and as a means of self-promotion.
The first case I have in mind involves a magician named William S Marriott. When not performing stage magic under the name Dr Wilmar, Marriott worked prolifically to debunk the methods of fraudulent mediums. He investigated a variety of paranormal phenomena including apparitions, spirit writing and thought reading.
Marriott was particularly concerned with spirit photography – the claim that images of the spirits of the dead could be captured on film.
Marriott's aim was to show that many of the supposedly genuine pictures could have been fakes.
While spiritualists such as Conan Doyle acknowledged that the photographs could be faked, they hoped that by strictly controlling the process of taking and developing the photographs, trickery could be scientifically ruled out. The argument was similar to Holmes' maxim: "When you have eliminated all which is impossible, then whatever remains, however improbable, must be the truth."
On 5 December 1921, Marriott set out to demonstrate that eliminating the impossible was much more difficult than Conan Doyle might have wished to believe. His plan was to invite Conan Doyle and three other witnesses to observe him as he took and developed photographs of the great writer. The committee carefully examined the camera and watched as Marriott took two photographs of Conan Doyle.
The witnesses, including an expert photographer, confirmed that they had watched Marriott closely and had seen no indication that the magician was doing anything other than going through the normal process of developing photographs.
Nevertheless, when the photographs were developed, the first included a translucent ghostly figure and the second featured a ring of dancing sprites, neither of which had been observed in the room when the photos were taken. (Marriott's choice to include fairies was presumably a dig at Conan Doyle's famous endorsement of the Cottingley Fairies photographs.)
The Sunday Express newspaper in the UK gleefully published both the images, emphasising the "ghost-like intruder" and Conan Doyle's "favorite fairies". They also published the witness statements along with Marriott’s assertion that he had, in fact used sleight-of-hand to manipulate the process.
Conan Doyle was characteristically affable about the proceedings and was pleased to have the fairy photo as a souvenir. He wrote: "Mr Marriott has clearly proved a point that a trained conjurer can, under close inspection of three critical pairs of eyes put a false impression upon a plate. We must unreservedly admit it."
But he continued to explain that his faith in spirit photography remained unshaken. In a perversely Sherlockian twist, he appealed to his readers to consider Marriott’s hands. "A conjurer," he argued, "has certain physical characteristics" such as "long, nervous artistic fingers". The mediums Conan Doyle trusted to produce real spirit photographs, in contrast had hands that were "short, thick and work stained".
The case of the "Masked Medium" represents an even more dramatic example of Conan Doyle's will to believe. Rather than witnessing images on plates, this scenario involved a live demonstration, orchestrated by a pair of performers.
The host of the event was actually a stage magician by the name of Percy Thomas Tibbles, who performed as P.T. Selbit, and the medium was an aspiring female illusionist named Molly Wynter, who had developed a specialisation in fraudulent medium tricks.
Wynter was presented as a genuine medium with Selbit acting as her manager. As Wynter identified the items in the locked box, and the ghost "materialised" in front of the audience, there was no indication that the audience were witnessing an elaborate trick – and it was only afterwards that the magician and illusionist explained their methods.
For the clairvoyance demonstration, Wynter’s veil had concealed not only her face but a wireless radio. Although the committee believed they watched the box with their objects the entire time, the box was switched so that Wynter was left holding a dummy box while an assistant in the other room opened up the actual box and broadcast its contents into her earpiece.
An acrobat, dressed entirely in black tights, scaled the building and entered through a window after the committee had completed their search of the room
The "materialisation", meanwhile, was accomplished not by ectoplasm, but by another secret assistant. An acrobat, dressed entirely in black tights, scaled the building and entered through a window after the committee had completed their search of the room. According to the magicians, "the ghost" was a bit of gauze coated with phosphorescent paint that the acrobat removed from their pocket and waved around the room.
To confirm their point, the illusionists repeated their demonstration before an even larger committee. And this is where Conan Doyle's reaction gets particularly interesting – he doubled down on his mistaken belief. Not only did he insist that what he had seen in the first séance was not what the magicians subsequently described, but he added that even if the second demonstration was accomplished by a trick, "there is nothing to show that the first séance was not genuine". He was sceptical of the sceptical explanation.
Perhaps the magicians really were psychic, he said, but were simply lying about their paranormal powers. He went on to rail against the media coverage of the demonstration. "It is time which will prove our cause," he wrote. "Time will also prove to those who have misrepresented us that they are playing with fire. They are not judging the Unseen. The Unseen is judging them." Reading those words now, I was struck that his sentiments, if not his coherent sentence structure, would not be out of place on a 21st Century Twitter feed.
Thinking about thinking
In a later TV interview, Conan Doyle tried to explain his views: "When I talk on this subject I'm not talking about what I believe. I'm not talking about what I think. I'm talking about what I know. There's an enormous difference, believe me, between believing a thing and knowing a thing, and talking about things that I've handled, that I've seen, that I've heard with my own ears. And always mind you in the presence of witnesses. I never risk hallucinations. I usually, in most of my experiments, have had six, eight, or 10 witnesses, all of whom have seen and heard the same things that I have done."
Given this reasoning, it is easy to take a dim view of Conan Doyle's own scientific detective skills, but I do think that he was genuinely a brilliant man. In addition to his literary accomplishments, he also worked as a real-life legal advocate, using Sherlock-esque techniques to exonerate and free several falsely convicted prisoners.
Conan Doyle’s reactions to these hoaxes are clearly problematic, but they are also an illustration of psychological phenomena known as "metacognitive illusions".
While it can indeed feel like our memories work like recordings, this idea is arguably a metacognitive illusion
"Metacognition" is the idea of thinking about thinking. By extension, metacognitive illusions occur when people hold mistaken beliefs about their own cognitive systems. We all tend to feel like we are experts about the nature of our own perceptions and memories. After all, we generally perceive things and remember things successfully throughout most of our day-to-day lives. However, in many cases our intuitions about our own cognitive systems can be surprisingly unreliable – we are not always nearly as observant as we think we are and our memories can be surprisingly malleable.
Surveys, for instance, reveal that more than half of the general public agree with the statement: "Memory works like a video camera accurately recording the events that we see and hear so that we can review and inspect them later." While it can indeed feel like our memories work like recordings, this idea is arguably a metacognitive illusion. According to our current theoretical frameworks, human memory is best understood as processes of reconstruction, rather than one of reproduction. That is to say that remembering an event is less like replaying a mental recording and more like composing a story.
The psychologist Elizabeth Loftus, for instance, has famously demonstrated that, in the process of recalling events, people can erroneously integrate imagined elements into their memories – even if these fictional memories can feel exactly like true memories. They can even be persuaded to believe they committed a crime. (Read more about how our memories cannot be trusted.)
Change blindness blindness
Another particularly striking metacognitive illusion Conan Doyle could have been guilty of is the phenomenon of change blindness blindness. But before we can explain this illusion, first we must describe the phenomenon on which it is based.
Change blindness refers to the phenomenon in which viewers fail to detect (sometimes surprisingly dramatic) changes to a visual scene. One way of demonstrating this effect is through a procedure called a "flicker paradigm" in which two very similar scenes alternate, with a few discrepancies between them. Detecting changes in a flicker paradigm is such a surprisingly difficult task that when a team of psychologists led by Ronald Rensink first tried to publish research on the topic in the mid-90s, their results were initially rejected by peer reviewers as impossible. In other words, the fact people could be change blind was so counterintuitive that even visual scientists were inclined to doubt the reality of the phenomenon.
Today, change blindness is an established part of cognitive psychology. Scientists have even introduced the term change blindness blindness to refer to the fact that people tend to be ignorant of their change blindness. But magicians like Marriott, Tibbles, and Wynter were exploiting such metacognitive illusions in their tricks long before psychologists became formally aware of such phenomena.
Audiences consistently overestimate their ability to detect the hidden methods behind magic illusions
There is now increased interest in using magic tricks in psychological experiments to illustrate these kinds of phenomena. A recent study by Jeniffer Ortega at the National University of Colombia in Bogota and colleagues specifically explored the role of metacognition in magic. Using a series of simple tricks with coins and playing cards, they showed that audiences consistently overestimate their ability to detect the hidden methods behind magic illusions.
People sometimes interpret these kinds of results as demonstrating that your perception is "broken" or your memory is "flawed". But they are really the result of an otherwise well-adapted cognitive system that is mostly effective in working with complex and confusing information every day.
It’s possible to appreciate the intricacies of human memory and perception, while considering the systems' limits and eccentricities, just as you can cherish Conan Doyle for his writing, even if you are not entirely comfortable with his more esoteric views. Reality is often stranger than we imagine.
Matthew Tompkins is an experimental psychologist and a professional magician. His new book The Spectacle of Illusion Is now available for purchase in stores and online.
Join one million Future fans by liking us on Facebook, or follow us on Twitter or Instagram.
If you liked this story, sign up for the weekly bbc.com features newsletter, called “The Essential List”. A handpicked selection of stories from BBC Future, Culture, Worklife, and Travel, delivered to your inbox every Friday.