The Cold War offered a perfect backdrop for spy fiction.  Masters of espionage like Ian Fleming’s James Bond, John le Carré’s George Smiley and Len Deighton’s Harry Palmer squared off against their communist counterparts in a global chess game with highly calculated moves and obvious goals, risks and rewards. Betrayals and deceptions complicated the matter, but the enemies were clear-cut. British intelligence officer Smiley against his KGB counterpart Karla. Us versus them. “During the Cold War, any reader opening a spy novel understood the handful of possible conflicts they would find,” says spy fiction author Olen Steinhauer. “It would be East v West, or the hero against political corruption – greedy Westerners destroying their own system.”

Readers of Cold War spy fiction were drawn into identifying double agents (as in Frederick Forsyth’s The Fourth Protocol) and fantasising about the effects of brainwashing (Deighton’s The Ipcress File). We were presented with spycraft on a human scale, without the satellite surveillance and mobile phone tracking systems  of today’s hi-tech thrillers. Characters suffered from psychological as well as physical stress. James Bond is on the verge of a nervous breakdown after his new bride is killed in You Only Live Twice. The Spy Who Came in from the Cold ends with a devastating portrait of the effect of years of duplicity on agents constantly justifying acts at odds with their moral values. But there were pleasures as well – dinners in fine restaurants, romantic dalliances, vicarious visits to far-flung parts of the world out of the range of most readers, including countries behind the Iron Curtain.

The end of the Cold War made it necessary to find new enemies. “When the Cold War ended, the genre lost a perfect adversary,” says spy fiction author Joseph Kanon. Writers could no longer depend upon an easy polarity. “Russia remains, but as a threatening kleptocracy,” says Steinhauer, whose Milo Weaver trilogy focusses on relations between the US and China, just one facet of a post-Cold War world that has many existential conflicts: “Terrorism existed as a subject during the Cold War, but now, of course, it's become a primary source of narrative conflict. And the complex relationships between Middle Eastern states and religious extremists make for more fictional fodder. Political corruption at home never goes away, but the American relationship to European states has become more complex,” he says. “The rise of electronic surveillance and the hero-status of the whistleblower... these things are all excellent subjects.” For his 2014 novel The Cairo Affair, Steinhauer worked almost in real time, chronicling Libya and post-Mubarak Cairo during the Arab Spring.

New world order

After 1989 it was time to rethink the spy game in fiction entirely. “I always wrote about people groaning under the moral weight of the Cold War and begging to get out,” John le Carré told the New York Times after the Berlin Wall came down. “I'm absolutely delighted to be presented with a new pack of cards.” And he dealt them out into plotlines featuring the international drug trade, the Russian mafia, money laundering and corporate corruption. In The Constant Gardener (2001), which is set in “dangerous, decaying, plundered, bankrupt, once-British Kenya,” he takes on pharmaceutical companies who used citizens in developing countries as guinea pigs in drug trials.

Soon le Carré was writing of the international war on terror and American operatives who justify torture and extraordinary rendition (as in his 2008 novel A Most Wanted Man), and their British allies. His character Toby Bell, a rising star in the British foreign service who seeks his country’s “true identity in a post-imperial, post Cold War world”, bears a resemblance to Edward Snowden. Frederick Forsyth also has followed the headlines. He set The Fist of War (1994) during the first Gulf War. The CIA, MI6 and Pakistan’s ISI battled al Qaeda in his 2006 novel The Afghan. And The Kill List (2013) features an internet-based ‘preacher’ who calls upon radical Muslims for assassinations of leaders in the US and Britain.

Some contemporary writers have chosen to set their books in the past, in the certainties of the Cold War, or even the period before it, rather than updating them for the uncertainties of the present day. Alan Furst‘s atmospheric thrillers (12 to date, most recently Midnight in Europe) take place in the 1930s and early ’40s, when Sovietand Western intelligence operatives joined forces to fend off the growing Nazi juggernaut.  Joseph Kanon focuses on the immediate postwar period, setting The Prodigal Spy (1998) and The Good German (2001) in 1945, “a pivotal time, the beginning of the world we live in now,” he says. For Istanbul Passage (2012) he turned to “a neutral city near the Balkans, a perfect listening post, a mecca for spies and a prime staging area for the Mossad in helping rescue trapped European Jews.”

The Cold War may prove to be an unparalleled inspiration for spy fiction.  But the essence of spy fiction hasn’t changed. Readers expect identifiable heroes defending against external foes. What has changed forever is the starkly black-and-white nature the Cold War lent the genre. The more ambivalent spies, says Steinhauer, “go about their jobs with a measure of anxiety, dealing with the moral burden – the subterfuge and lying the job requires.”  Readers have become accustomed to seeing the flaws in our own system, and in ourselves, thanks to sceptical portraits by these masters of Cold War spy fiction. This era of clearly defined heroes and villains has given way to the blurred lines, shifting allegiances and ambiguity of today.

If you would like to comment on this story or anything else you have seen on BBC Culture, head over to our Facebook page or message us on Twitter.