Big Data: Are you ready for blast-off?

Japan rocket blasting off A recently launched Japanese-US satellite will monitor global rainfall, helping meteorologists use big data analytics to predict storms around the world

As Technology of Business begins a month-long series of features on the theme of Big Data, we kick off with a Q&A backgrounder answering some of those basic questions you were too afraid to ask.

Technology of Business
What is big data exactly?

Good question. After all, we've always had large amounts of data haven't we, from loyalty card schemes, till receipts, medical records, tax returns and so on?

As Laurie Miles, head of analytics for big data specialist SAS, says: "The term big data has been around for decades, and we've been doing analytics all this time. It's not big, it's just bigger."

But it's the velocity, variety and volume of data that has merited the new term.

So what made it bigger?

Most traditional data was structured, or neatly organised in databases. Then the world went digital and the internet came along. Most of what we do could be translated into strings of ones and noughts capable of being recorded, stored, searched, and analysed.

There was a proliferation of so-called unstructured data generated by all our digital interactions, from email to online shopping, text messages to tweets, Facebook updates to YouTube videos.

Man checking phone before networked society poster As the number of mobile phones grows globally, so does the volume of data they generate from call metadata, texts, emails, social media updates, photos, videos, and location

And the number of gadgets recording and transmitting data, from smartphones to intelligent fridges, industrial sensors to CCTV cameras, has also proliferated globally, leading to an explosion in the volume of data.

These data sets are now so large and complex that we need new tools and approaches to make the most of them.

How much data is there?

Start Quote

Early adopters of Big Data analytics have gained a significant lead over the rest of the corporate world”

End Quote Bain & Company

Nobody really knows because the volume is growing so fast. Some say that about 90% of all the data in the world today has been created in the past few years.

According to computer giant IBM, 2.5 exabytes - that's 2.5 billion gigabytes (GB) - of data was generated every day in 2012. That's big by anyone's standards. "About 75% of data is unstructured, coming from sources such as text, voice and video," says Mr Miles.

And as mobile phone penetration is forecast to grow from about 61% of the global population in 2013 to nearly 70% by 2017, those figures can only grow. The US government's open data project already offers more than 120,000 publicly available data sets.

Where is it all stored?

The first computers came with memories measured in kilobytes, but the latest smartphones can now store 32GB and many laptops now have one terabyte (1,000GB) hard drives as standard. Storage is not really an issue anymore.

NSA data centre in Utah The US National Security Agency has built a huge data centre in Bluffdale, Utah - codenamed Bumblehive - capable of storing a yottabyte of data - that's one thousand trillion gigabytes

For large businesses "the cost of data storage has plummeted," says Andrew Carr, UK and Ireland chief executive of IT consultancy Bull. Businesses can either keep all their data on-site, in their own remote data centres, or farm it out to "cloud-based" data storage providers.

A number of open source platforms have grown up specifically to handle these vast amounts of data quickly and efficiently, including Hadoop, MongoDB, Cassandra, and NoSQL.

Why is it important?

Do you know your big data numbers?

1,000 bytes = one kilobyte (kB)

1,000 kB = one megabyte (MB)

1,000 MB = one gigabyte (GB)

1,000 GB = one terabyte (TB)

1,000 TB = one petabyte (PB)

1,000 PB = one exabyte (EB)

1,000 EB = one zettabyte (ZB)

1,000 ZB = one yottabyte (YB)

(Figures are decimal, not binary)

Data is only as good as the intelligence we can glean from it, and that entails effective data analytics and a whole lot of computing power to cope with the exponential increase in volume.

But a recent Bain & Co report found that of 400 large companies those that had already adopted big data analytics "have gained a significant lead over the rest of the corporate world."

"Big data is not just historic business intelligence," says Mr Carr, "it's the addition of real-time data and the ability to mash together several data sets that makes it so valuable."

Practically, anyone who makes, grows and sells anything can use big data analytics to make their manufacturing and production processes more efficient and their marketing more targeted and cost-effective.

It is throwing up interesting findings in the fields of healthcare, scientific research, agriculture, logistics, urban design, energy, retailing, crime reduction, and business operations - several of which we'll be exploring over the coming weeks.

Thai farmer works in rice field By analysing weather, soil, topography and GPS tractor data, farmers can increase crop yields

"It's a big deal for corporations, for society and for each individual," says Ralf Dreischmeier, head of The Boston Consulting Group's information technology practice.

Can we handle all this data?

Big data needs new skills, but the business and academic worlds are playing catch up. "The job of data scientist didn't exist five or 10 years ago," says Duncan Ross, director of data science at Teradata. "But where are they? There's a shortage."

And many businesses are only just waking up to the realisation that data is a valuable asset that they need to protect and exploit. "Banks only use a third of their available data because it often sits in databases that are hard to access," says Mr Dreischmeier.

"We need to find ways to make this data more easily accessible."

Businesses, governments and public bodies also need to keep sensitive data safe from hackers, spies and natural disasters - an increasingly tall order in this mobile, networked world.

Who owns it all?

That's the billion dollar question. A lot depends on the service provider hosting the data, the global jurisdiction it is stored in, and how it was generated. It is a legal minefield.

Facebook logo Facebook's logo - created using photos of its global users - adorns the wall of a new data centre in Sweden - its first outside the US. But who has rights to all the data?

Does telephone call metadata - the location, time, and duration of calls rather than their conversational content - belong to the caller, the phone network or any government spying agency that happens to be listening in?

When our cars become networked up, will it be the drivers, owners or manufacturers who own the data they generate?

Social media platforms will often say that their users own their own content, but then lay claim to how that content is used, reserving the right to share it with third parties. So when you tweet you effectively give up any control over how that tweet is used in future, even though Twitter terms and conditions say: "What's yours is yours."

Privacy and intellectual property laws have not kept up with the pace of technological change.

More on This Story

The BBC is not responsible for the content of external Internet sites

More Business stories

RSS

Business Live

  1.  
    12:17: Tesco store closures
    Tesco

    More on the Tesco store closure story. In total, 2,000 staff are affected. Express and Home Plus stores will close on 15 March. Tesco Metros and Superstores will close on 4 April.

     
  2.  
    Tesco store closures Breaking News

    Tesco have named the 43 stores to shut: 18 express stores, 12 metro, seven superstores and six homeplus outlets. "We will continue to serve our customers through other local stores and our dotcom service," said boss Dave Lewis.

     
  3.  
    11:51: Petrol prices
    petrol

    Oh er. Petrol price drops have levelled off somewhat, according to Experian Catalyst's figures. Admittedly, these are daily numbers, so they are likely to jump around a bit. The average price for unleaded yesterday was 106.69p. The day before: 106.66p. A week ago it was 107p and 112.08p on 1 January.

     
  4.  
    11:35: Greek markets
    ase

    Greek bond yields have risen again, meaning their value in the market has dropped and investors are considering them riskier. 10-year bonds now offer investors a 10% return. Shares on Athens's main market (pictured) dropped again, with banks leading the losers. Eurobank Ergasias, National Bank of Greece and Piraeus are all down 19 to 20%.

     
  5.  
    11:20: Companies House blunder

    The High Court has order Companies House to pay £9m to Taylor & Sons Ltd, a Cardiff based engineering firm that it mistakenly reported as having been wound up, the Daily Telegraph reports. In fact, it was another, entirely unconnected, company - Taylor & Son Ltd - which had actually gone to the wall. By the time Companies House, tried to correct its mistake three days later, it was already too late to save the business.

     
  6.  
    11:04: No rush to raise interest rates
    The Bank of England

    The Bank of England is in no rush to hike interest rates and when it does, the increase could be as slow as half a percentage point a year, chief economist Andy Haldane has said."We are in no rush to raise rates, the recovery is taking hold nicely. The last thing we want to do is knock the stuffing out of that," Haldane told the Daily Post. They are historically low and they won't stay like that forever, but when that rise comes it is going to be very gradual. It could be half a percent a year for several years," he added.

     
  7.  
    10:47: Ikea profits flat
    The new IKEA store in Edmonton

    Ikea has reported flat full-year net profit as a result of a €200m contribution to an employee loyalty programme, and a growing number of staff taking part in a bonus programme, for which it booked a €98m cost. It says the European market continued to improve. Net profit was €3.3bn to the end of August 2014.

     
  8.  
    President Obama India visit Via Email Simon Atkinson Editor, India Business Report
    Mahindra

    Anand Mahindra, chairman of Indian conglomerate Mahindra Group has been telling the BBC about being in talks with US President Barack Obama and Indian Prime Minister Narendra Modi in Delhi. And he was impressed with the informality: "It was literally talking over a cup of tea," he said. "President Obama took notes and referred back to specific points people made, so I know he wasn't just doodling".

     
  9.  
    10:16: African bond risk

    A strengthening dollar could sting some African countries after a spree of bonds being issued in the currency, according to a think tank. Judith Tyson, senior researcher at the Overseas Development Institute (ODI), said the notional losses in terms of higher interest and capital repayments could total $11bn in the event of a slump of 35% in African currencies against the dollar, Reuters reports. Selling bonds in hard currencies is a popular practice to attract foreign investors.

     
  10.  
    10:00: Greece stops privatisations

    Shares of Greek energy firm PPC and Greece's biggest port, Piraeus Port Authority, have fallen by over 7% after the new government's declaration that it was freezing privatisation plans. Shares of Piraeus Port Authority were down 8.9% at €9.83. At the first cabinet meeting of the new government Greek Prime Minister Alexis Tsipras has said Greece will not default on its loans.

     
  11.  
    09:45: Greece stops privatisations

    The Greek government will freeze plans to privatise the country's dominant energy firm PPC, Energy Minister Panagiotis Lafazanis has told Greek television today. The previous government had passed legislation last year to spin off a chunk of PPC and privatise it as part of its efforts to liberalise its energy market under the terms of its IMF-EU bailout A planned sale of a 67% stake in the State owned Piraeus Port Authority has also been shelved.

     
  12.  
    09:28: Russian plan
    People walk in the Red Square

    The Russian government unveiled a 2.34 trillion-rouble ($34.8bn, £22.9bn) plan to tackle its economic and financial troubles prompted by a collapse in oil prices and Western sanctions over the Ukraine crisis. In the plan, the government said it would collect proposals for creating a 'bad bank' for problematic banking assets. It would also provide state development bank VEB with 300 billion roubles from the National Wealth Fund.

     
  13.  
    09:13: Greece euro exit? BBC Radio 4

    Greece leaving the euro wouldn't do irreparable damage to the eurozone, Carl Bildt tells Today. It is up to Greece whether it wants to stay in the eurozone or not, but the currency bloc itself is here to stay, he adds. "Leaving the eurozone is not going to sort out Greece's problems," he says.

     
  14.  
    08:58: Greece euro exit? BBC Radio 4

    "Greece evidently wants to stay in the euro," Carl Bildt, tells Today. There will be discussion and there will be division between the Greek government and the rest of Europe, he adds. "The rest of Europe already feels it has helped Greece massively and Greece has turned things around slightly, it's not a hopeless case but now, suddenly, they want to spend and go back to irresponsible policies," he says.

     
  15.  
    08:46: Newspaper review
    papers

    The Times reports doctors can now approve Eculizumab, a treatment for a rare kidney condition. It will cost the NHS £82m a year for the 200 people with Atypical hemolytic-uremic syndrome. The FT leads on Apple's performance. The Wall St Journal dissects the new Greek cabinet and the Daily Telegraph reports that the drop in petrol price is as good as a 1p tax cut.

     
  16.  
    08:32: Alibaba report
    alibaba

    Chinese regulators have accused online shop Alibaba of allowing sales of fake and poor-quality goods in a report that was withheld until now to avoid disrupting its US stock market debut. China's industrial regulator said Alibaba allowed unlicensed dealers to use its services and failed in consumer protection.

     
  17.  
    08:22: Market update

    Tokyo stocks reversed early losses to close 0.15% higher as a weak yen pushed the market into gains. The Nikkei 225 index at the Tokyo Stock Exchange, which fell 0.86% at the open, added 27.43 to end at 17,795.73. The benchmark Hang Seng Index in Hong Kong gained 54.53 or 0.22% to 24,861.81.

     
  18.  
    08:13: Nintendo
    mario

    Japanese videogame maker Nintendo said its April-December profit rose six-fold to $504m as a weaker yen boosted its bottom line and make up for slowing sales. It won't make as many sales for the full year as previously hoped, though.

     
  19.  
    08:05: Greece euro exit? BBC Radio 4

    Former Swedish prime minister Carl Bildt tells Today it is not just the Germans that oppose a renegotiation of Greece's debt repayments. He points to Finland which holds elections later this year and has some problems in its economy. He says it is very hard for Finnish MPs to tell the public they have to accept some cutbacks in public spending but Greece doesn't. "That doesn't really fly," he says.

     
  20.  
    07:48: EDF disconnects nuclear power plant
    A view of Heysham Nuclear Power Station

    EDF Energy has said it disconnected Heysham's 1-2 nuclear power plant at the Lancashire site from the electricity grid on Tuesday. "Unit 2 at Heysham 1 power station was taken offline on 27 January 2015," an EDF spokesman said in an emailed statement to the Reuters newswire. EDF tells the BBC it's an issue with a turbine. So it's a non-nuclear part of the plant. Should take "a few" days to sort.

     
  21.  
    07:34: AG Barr results
    bru

    Fizzy drink maker and manufacturer of Irn Bru said it delivered a "robust performance" and should hit analysts' targets for the year. It had 5% growth in the fourth quarter, "well ahead of the total soft drinks market."

     
  22.  
    07:26: Greece euro exit? BBC Radio 4

    Greece should tell Europe it will not pay its debt and will exit the eurozone, Lord Desai tells Today. He says the country has to decide whether it wants "misery now or misery forever". He suggests that has to be the starting point of any negotiations with the Troika - the IMF, ECB and EU. He adds no country has ever paid debts like this in the past. Germany did not pay all its reparations, he says. "The international community has to recognise you can't put such a incredible burden on a people regardless of whose fault it was," he adds.

     
  23.  
    Via Twitter Rory Cellan-Jones Technology correspondent
    apple

    tweets: Amazing Apple quarter: record $18bn profit, 74m iPhones - most profitable product in history? But iPads disappoint

     
  24.  
    06:57: Greek euro exit? BBC Radio 4

    Lord Desai says most of the debt Greece owes is to public bodies such as the European Union and IMF. Greece cannot continue to pay off its debts for the next 20 years, he adds, and Greece and Germany have to decide whether they can afford for Greece to leave the eurozone. Anne Richardson of Aberdeen Asset Management points out £8bn of bank deposits have left Greek banks since November because investors see a so-called Grexit as having come "one step closer".

     
  25.  
    Via Twitter Stephanie McGovern Breakfast business reporter
    port

    tweets: Morning from the Port of Tyne - where today I'm talking about exports. #economy

     
  26.  
    06:44: Greek euro exit? BBC Radio 4
    Greece"s Prime Minister Alexis Tsipras is accompanied by associates

    Now that the Greek election has been won by Syriza, thoughts have begun to turn to negotiations over the country's debts. Greece could request to pay no interest on those debts for about five years, Lord Desai, economist and chairman of the Official Monetary and Financial Institutions Forum, tells Today. "That would save them about 4% of GDP," he says. But, he adds, there "really is no human way that Greece can pay the debt without ruining at least one generation's future."

     
  27.  
    06:31: GDP growth Radio 5 live

    Anne Richards, chief investment officer of Aberdeen Asset Management is the markets guest on Wake Up to Money. "You have to be a wee bit careful with quarterly numbers as they are subject to a lot of revision," she says. "The overall picture for the year was reasonably positive." Low construction growth was "a bit worrying." Strong sterling is a drag on GDP growth. Reliance on services rather than making things is also a challenge, she says. More engineers are needed.

     
  28.  
    06:20: Apple profit Radio 5 live
    The Apple logo

    The biggest quarterly profit ever for a company: $18bn, has been posted by phone pedlars Apple. Daniel Eran Dilger who writes for AppleInsider tells Wake Up to Money. Apple makes a load of margin from its high-end phones. They also make a lot when you break your power cable and have to splurge £65 on a new one, as presenter Adam Parsons learned earlier this week.

     
  29.  
    06:12: Services growth Radio 5 live

    More from Greg Madigan, the boss Subway UK and Ireland on Wake Up to Money. Hospitals, service stations and forecourts, or "non-traditional locations" are a big area of growth for the firm, he says. He used to be an air traffic controller, he adds.

     
  30.  
    06:01: Services growth Radio 5 live

    Services is what's propping up GDP growth, we learned yesterday. Greg Madigan, the boss Subway UK and Ireland is on Wake Up to Money. "The price of oil has come down putting more money in peoples pocket... one of the things that benefit from more discretionary spending is food retail," he says. They have 2,000 stores in the UK and Ireland now.

     
  31.  
    06:00: Howard Mustoe Business reporter

    Good morning. Keep your thoughts on today's news rolling in via email bizlivepage@bbc.co.uk and on twitter @BBCBusiness

     
  32.  
    06:00: Matthew West Business Reporter

    Morning everyone. In case you missed it EDF became the last of the "Big Six" energy suppliers to cut its gas prices last night. And US tech giant Apple reported the largest quarterly profit in corporate history. Today sees trading updates come from Brewin Dolphin, Johnson Matthey, Sage and Anglo American. We'll bring you those numbers and more as we get them.

     

Features

From BBC Capital

Programmes

  • Virtual courtroomClick Watch

    The 'forensic holodeck’ system that recreates crime scenes as 3D virtual worlds

Copyright © 2015 BBC. The BBC is not responsible for the content of external sites. Read more.

This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.