Big Data: The Basics

The IT landscape is constantly changing, with a major breakthrough announced every few weeks and equipment becoming apparently obsolete in just a year… at least that’s how Apple wants you to see things. It’s often a bamboozling world too, as IT managers have to sift their way through all the latest jargon and cornucopias of new product sets.

With this in mind, we asked Glyn Heath, managing director at IT consultancy Centiq, to tackle one of the subjects on everyone’s mind – Big Data.  Centiq is dedicated to delivering smarter in-house and cloud infrastructure, helping blue chip, mid-range and public sector organisations manage their IT estates. Heath looks at the basics, explaining what Big Data is and what it’s good for.

Useful terabytes

What is Big Data, in a nutshell?

Big Data refers to sets of data that are so large and complex that traditional database management tools struggle to capture, store, analyse and share this information.

‘Big Data analysis’ is the real-time analysis of large quantities of an organisation’s data – which is then used to provide insights into how well a company performs.

Where has Big Data come from?

Big Data has materialised through the vast amount of information modern companies generate in their business processes.

Big Data analytics has emerged through advances in software technology that have enabled this mass of data to be processed in real-time.

The trend to assemble larger data sets is due to the additional information derivable from analysis of a single large set of data, as opposed to smaller sets with the same total amount of data. Analysing these sets of aggregated data as a whole is faster and allows drawing more precise conclusions.

How big is it, really?

Whether information can be classed as Big Data depends on the IT capabilities of the organisation managing the information. For some, handling hundreds of gigabytes of data for the first time would be considered a Big Data challenge. For others, it may take tens or hundreds of terabytes before the term Big Data would apply.

How much money is there in Big Data analytics?

According to market research firm IDC, thanks to Big Data, analytics is estimated to be a $51 billion business by 2016.

What are the benefits that Big Data analytics can bring?

Businesses can use Big Data analytics to determine whether key promotions, product lines, sales territories and customers are profitable enough – or even sustainable.  For example, the ability to process data in real time can drastically affect the web-based shopping experience by making highly targeted suggestions ‘in-basket’.

A retailer being able to respond to sales data on the fly might use this to alter TV or radio advertising the same day.

Another example would be a Fast Moving Consumer Goods (FMCG) business tuning its production and stocking logistics with greater accuracy and frequency to improve the efficiency of the supply chain.

New and unexpected knowledge from analytics engines isn’t just a cutting edge development, it could be a real lifeline for businesses seeking to innovate or simply drive costs from unprofitable or outdated product in a challenging economy, where capital budgets will remain tight for years.

Analyse This

Why is working with Big Data complicated? What are the challenges and problems?

Big Data is not a miracle cure for slow and sluggish core systems that frequently provide limited, out-of-date information.  

Until companies understand how to optimise and accelerate their reporting systems, they will be restricted in their ability to utilise Big Data analytics.

What kinds of tools do you need to work with Big Data?

Companies need to ensure that they do the groundwork before they can build advanced analytics systems. A lot of firms have started laying the foundations with data warehousing, speeding up and adding sophistication to reporting out of ERP and other systems.

Tools such as SAP Business Warehouse Accelerators have enabled wider company management to use data to improve decision-making and speed up responsiveness. For many firms, this can be a foundation for more advanced analytics capabilities.

Are Big Data benefits restricted to big firms?

No – valuable information, on business aspects ranging from company processes to customer behavior, is hidden within the data generated by companies. Accessing this information, and obtaining relevant results at speed, can give businesses a real edge in markets weighed down by the recession. Being able to understand the workings of your company’s infrastructure, or react to what consumers are looking to buy, can provide genuine insight into profitability and improve ROI.

However, until the technology becomes ‘mainstream’ and is widely adopted, the costs could remain prohibitive. That said, even for relatively small businesses wishing to analyse modest amounts of data, the return on investment will be driven by the specific use-cases and might still stack up commercially.

Could small-to-medium sized businesses run their own big data projects?

For mid-range firms with ambition and the ability to innovate, there are many open-source solutions available that would minimise the cost of acquisition of the technology. However, unless the business is fully behind this kind of initiative and the technology team is confident of its capabilities, this could turn out to be a risky and ultimately expensive foray.

How well do you know open-source software? Take our quiz!