Bad guys v the data defenders: Let battle commence
Big data analytics is making it easier to spot the bad guys looking to infiltrate business defences.
And these days, businesses need every weapon at their disposal, as bedroom hackers give way to organised criminal gangs.
Lose your data and you can lose your reputation, customers, and even your business.
This week, US retail giant Target Corporation, which suffered a massive theft of customer data last year, offloaded its chief information officer, Beth Jacob, as part of a major overhaul of its security practices.
Stolen details of about 360 million customer accounts are now available on cyber black markets, according to security firms.
External data leaks affected more than 160 million people in 2012, according to KPMG's Data Loss Barometer, a rise of 40% on the year before.
And hacking accounted for 67% of the data loss by number of incidents.
Small needle, big haystack
"Big data is about pushing the needle out of the haystack irrespective of how big the haystack has become or how small the needle is," said Gordon Harrison, an industry consultant at data analytics specialist SAS.
Haiyan Song, vice-president of security at big data analysis firm Splunk, said analysing reams of data to spot security breaches had become essential because of the changing tactics of the criminals.
Hi-tech thieves have changed their tactics because security companies have got so good at spotting malware. Instead, the bad guys are relying on more subtle tactics and strive to slip inside a company unnoticed.
Big data analysis tools could help pick them out of the crowds of data, said Ms Song.
That tactic of seeking to trick people into giving them access has been helped by the fact that the digital perimeter of a company is now much harder to pinpoint.
In the good old days, said Ms Song, such borders had been easy to identify. Set up the firewalls, email gateways and keep your virus signatures up-to-date, and you had a good chance of staying safe.
Now? Not so much.
"The borders have been taken out of any and every enterprise," said Ms Song.
The borders are much less easy to define thanks to the internet, which lets customers query back-office systems via a website, the ties that exist between trading partners, and innovations such as Bring Your Own Device.
"Companies have got better at hardening their perimeters so the advanced threat actors are going after the extended borders of the company such as the point-of-sale systems," she said.
Keeping secure, protecting the back office and the databases and stopping the bad guys getting at customer data is all about looking for anomalies in the voluminous stream of data that all the parts of a computer network spit out.
"Before now, without big data analytics, it would be hundreds and hundreds of man hours trawling through the application to spot those exceptions," said Mr Harrison, from SAS.
Those strange bumps in the data could reveal the advanced threats - the nasty ones involving criminals scouring social media for information they can use to make phishing emails more plausible.
For example, if you play squash, row for a local club or go rambling, and are a senior executive, there's a chance that personal information will be scooped up by the bad guys. They use it to make their next phishing email look like it comes from someone you know who is writing about a subject you deeply care about.
These advanced threats also often use undocumented software vulnerabilities so the malware signatures do not pick them up. No wonder that the average time it takes companies to detect one of these threats can run to months.
The most far-sighted companies did not just use the big data stream from their networks as a way to keep their data safe, said Martin Borrett, director of security systems at IBM.
That information could also reveal the processes underpinning the way a company works, he said.
"Big data can create a culture in which business and technology leaders join forces to realise the value in the data," he said. "Its insights can enable all employees to make better decisions, deepen customer engagement, and optimise operations."
But, he said, companies that took this step had to make sure they protected what could be of great value in a competitor's hands.
Protecting access to it was key, but that did not mean locking it all away, he said.
"It is important to understand your data," said Mr Borrett. "It is only through understanding the nature of it that you can work out the appropriate level of security to apply."
And the sheer amount of data in the average big data store introduces another potential security risk - the cloud.
Uploading all your information to a cloud and running analytics on it might save companies cash, but they have to take steps to ensure rivals and others cannot get at it, too.
To help protect the information, IBM has developed a technique known as homomorphic encryption that lets work be done on data even though the underlying information is obscured.
That need to keep certain data away from prying eyes has grown in the wake of revelations about the extent of surveillance by the US National Security Agency and GCHQ in the UK.
Many firms are now turning to companies such as CipherCloud to ensure that when information passes out from the company borders it stays locked away from prying eyes.
Freeing it from the company data centre could prove to have other benefits, said Pravin Kothari, the company's founder and chief executive.
"The data can be used once or many, many times over," he said, and could spur other departments to get involved with any big data project.
"It gives you much more visibility and control over that information," he said, adding that control was the key to security.