Criminals brew their plans online and in texts. Authorities want to stop this – but how can governments keep the public safe and also respect its privacy?

Remember that picture you sent to your family of your children playing in the paddling pool? Or that private text you sent to someone trusted? Or when you searched for medical advice? 

Special series: Cyber-hacks

This week, the BBC is taking a close look at all aspects of cyber-security. The coverage is timed to coincide with the two biggest shows in the security calendar - Black Hat and Def Con.

Follow all the coverage via this link

Then, guess what: those messages and websites you visited will be stored and could potentially be obtained by criminals. What’s more, as soon as these messages are sent, their metadata (who we speak to, where we were, when it took place and how long we spent talking) and the content of communications could potentially be read by government agencies with a warrant.

When it comes to the services you're connected to the internet, the Investigatory Powers Act (known derisively to some as the Snooper’s Charter) allows government agencies, under certain circumstances, to access those records. What's more, a technical capability regulation was leaked in May to the Open Rights Group, a civil rights group. Such a notice would legally compel a telecommunications firm to record all of the communications by the target(s) named in the warrant, and to transmit this information, in near real-time, in a readable format (if it's already in a readable format).

Following the recent terror attacks in London and Manchester, Prime Minister Theresa May reiterated her intention to “regulate cyberspace to prevent the spread of extremist and terrorism planning.”

This is to help protect us from future acts of terrorism – and disclosing intercepted content in real time isn't new – but is it an intrusion into our private lives?

Safety vs privacy – which is more important?

In the UK, the vast majority of the population support the prevention of terrorism: in 2010, a survey revealed that nine in 10 are happy with a proportionate loss of privacy in certain circumstances, such as full-body scanners in airport security checks. And in 2015, another survey showed that double the amount of Americans were concerned the government wasn’t doing enough to fight terrorism than the amount of Americans who were concerned with losing certain civil liberties in the process.

The IP Act first came into effect on 29 November 2016. From that point, all telecommunication services whose providers were given notice by the Secretary of State were compelled to store records of certain electronic communications, and record people’s internet connection records, as well as, when served with a warrant, allow the government access to this information.

But what if one day, information regarding every app, email, instant message, text, podcast, video and Skype call might be recorded and stored by service providers?

Whilst the actual content of these communications is not stored, it is the associated metadata that is recorded. For intelligence agencies, this information can offer far more insight than the actual content of the communication.

The Investigatory Powers Act was enacted in order to protect against terrorist attacks - and indeed, all crime - but the wide-reaching surveillance powers enshrined in this legislation has been likened by some to someone accessing your phone. “Under the guise of counter-terrorism, the state has acquired totalitarian-style surveillance powers and this is the most intrusive system of any democracy in history,” says Silkie Carlo, senior advocacy officer with Liberty, a UK-based human rights watchdog group.

There is the argument of “Nothing to hide, nothing to fear”, in support of what some might consider mass surveillance. However, whilst we may have nothing to hide, we still have the right to a private life. 

Unfortunately, in recent years, some have come to view ‘privacy’ as having something illicit to hide. Privacy may be a significant concern to people who are engaged in legal, but immoral activities, but all internet users should be concerned about the security of their data and safeguards against its misuse.

Under the guise of counter-terrorism, the state has acquired totalitarian style surveillance powers and this is the most intrusive system of any democracy in history - Silkie Carlo

Others have argued that internet companies, like Facebook and Google, already have access to much of this data. However, we consent to this access in exchange for free use of their services, through accepting their terms and conditions.

The Investigatory Powers Act replaced parts of the Regulation of Investigatory Powers Act 2000 and expanded the surveillance powers to include all forms of communications. “It is legislating their existing powers so they could be regulated and there could be some oversight of what they are doing,” says Monica Horten, a visiting fellow for the London School of Economics.

Potential blanket surveillance formalises intelligence services’ ability to store and analyse the communications of a population for any possible crimes being committed - major ones that threaten national security. Through automated or manual analysis of our online behaviour, the intelligence services will be able to predict potential terrorists plotting new attacks.

“The interception [of online communications] exists, it is just a matter of putting things into formality, which is better, because you can have oversight and some level of judicial scrutiny,” says Yair Cohen, social media solicitor and author of The Net is Closing: Birth of the e-police. “Nobody is complaining when terrorists are stopped because of interception.”

However, blanket surveillance that harvests everyone’s data presents its own problems.

Investigating criminals’ online info needs to be more precise

“The Draft Regulation Capability Notice does not tell us anything we didn’t know anyway when the bill went through,” says Ross Anderson, professor of security engineering at Cambridge University. “They explicitly give [government agencies] the power to serve arbitrary secret orders on technology firms. This is going to lead to serious trouble.”

One of the key aspects of the Investigatory Powers Act is that internet connection records will be stored for a year. Unlike browsing history, which records each webpage we have visited, the internet connection records are a history of each website visited.

This record of online behaviour provides telling information of who the subject is and what they do. Repeated visits to the NHS website would indicate a person with medical concerns, whilst visiting a particular bank indicates where they keep their money. “The amount of data they are collecting is quite a lot and the picture they can build up using the metadata is quite significant,” says Horten. “They can build up a picture of you and your lifestyle.”

The details of the security requirements are mandated in the technical capability notices, which are all confidential. However, the recent hacks of TalkTalk and Yahoo, which saw millions of accounts leaked, highlighted failings within their network security. There have also been incidents where government laptops, containing highly sensitive information, have been lost.

A potential repository of all our personal communications and details, in any country, will naturally become a target for hackers.

Private info motherlode

Using any kind of stored information in any context or country, criminals could be able to conduct increasingly personalised scams, targeting us based on our online habits. Scammers already pose as major banks and large companies, such as BT and Microsoft. Access to internet connection records might enable criminals to perform increasingly targeted scams against individuals, using information such as their favourite online stores or which charities they support, in order to build trust and appear legitimate.

The wide-reaching surveillance powers enshrined in this legislation has been likened by some to someone accessing your phone

With real-time access, scammers could be able to accurately reference recent online activities, making their fraudulent emails appear all that more legitimate.

The sheer of amount of data that can now be collected is colossal. As of 2015, there were 65 million people living in the UK. As 90% of the UK population is using the internet, one can do the maths and might assume that this means there will be 58.5 million records. “Even mathematically, it is an absolute certainty that you will generate false-positives if your starting point for surveillance is the entire population,” says Carlo.

The Investigatory Powers Act allows a broad number of government agencies audited access to the stored personal data. These range from agencies like the Metropolitan Police and the security services, through to the Food Standards Agency. There is also the question of who, within these agencies, would be able to view this information and how access to the information would be regulated and audited for potential misuse.

Edward Snowden, who leaked classified information from the National Security Agency (NSA) in 2013 was not an employee of the NSA, but a sub-contracted employee from Dell. Despite his status as an external subcontractor, Snowden was nonetheless able to leak the largest number of classified government documents ever in the history of the intelligence services. If Snowden was able to access this information, it is fair to ask what guarantees the public now have that similar incidents will not occur with their private data.

The Investigatory Powers Act also compels telecommunication companies to install government equipment at their premises, for the purposes of interception (also nothing new). Whether this equipment comes under the judicial review is unclear. “It is like the difference between the police getting a search warrant to search your home, and putting a policeman in to live as one of the family,” observes Horten.

In order to regulate its use, the Investigatory Powers Act includes a judicial double-lock, where a warrant is required for access, followed by judicial review. However, Liberty argues that this is only a single-lock. “The Judicial commissioner has to sign off the secretary of state or the police constables on a judicial review basis, which means they are signing off a warrant on the basis that the correct procedure has been followed, rather than signing off a warrant on the basis of the facts,” says Carlo.

The European Court of Justice has argued that the indiscriminate collection of data was against EU law. Liberty has also gained high court approval to challenge the Investigatory Powers Act.

“When you have declared yourselves to be God – which is what the Investigatory Powers Act does – there are no powers left to take,” says Anderson.

--

Join 800,000+ Future fans by liking us on Facebook, or follow us on Twitter.

If you liked this story, sign up for the weekly bbc.com features newsletter, called “If You Only Read 6 Things This Week”. A handpicked selection of stories from BBC Future, Earth, Culture, Capital, and Travel, delivered to your inbox every Friday.