Tackling The Smart Meter Data Avalanche

Understanding the data

Socha argues that utilities need guidance on how to avoid getting drowned in data and also how to exploit it. “It’s the one where they don’t have a friend just now, they don’t have a trusted adviser to turn to who can say, we’ve done this before.”

The solution, as far as Socha is concerned, is in data warehouses. “We have all of this data coming in from smart metering into an MDM [meter data management solution], but it’s useless there, unless we can integrate it with our billing system, our customer relationship system, our asset systems etc.,” he said.

Data warehouses allow data to be offloaded from the operational systems, providing both space and scalability. Companies can consolidate disparate data sources into a single efficient engine, making it easier to retrieve and analyse data.

Teradata, along with other firms such as IBM,  is also preparing for future growth in smart metering infrastructure, offering what it calls ‘linear scalability’. “This problem is only going to get bigger. Our pitch to the utilities is, we can come in with our entry level appliance just now, and as the challenges come from more smart metering – from more smart grid remote data, from having to deliver the kind of analytics that will allow you to go to this active smart grid – Teradata is entirely capable of growing with you.”

Linear scalability

While companies such as Oracle and IBM also offer solutions to combat the data avalanche from smart metering, Socha believes that Teradata’s linear scalability gives it a competitive advantage. “We’re data warehouse people. IBM and Oracle have a portfolio wider than the eye can see. We are a company of 7,000 people and this is all we do,” he said.

According to Socha, however, the solution itself is just the start. Utilities companies will also need guidance to help them make the most of the data they collect. “The consultancy is as big and as important as the box we bring along on the back of a lorry,” he said.

With so much personal data flying over the network, two areas that needs to be handled with delicacy are privacy and security. According to Socha, the privacy of data is a policy issue. It is between the individual and the utility to agree on the granularity of data that is taken. Security, on the other hand, is the sole responsibility of the utility.

However, he believes that the systems are already in place to deal with data security, so this will not be too much of a challenge for utilities.

“The security of massive amounts of data is already available, but there will also be archiving policies and disposal policies, and they exist already. The utilities businesses have to store particular kinds of data for a certain amount of time, have to get rid of data after a certain amount of time and have to be able to demonstrate that they have done so,” he explained

“So those processes and procedures are already in place. They may have to be refined for the new data model, but it’s nothing particularly new and frightening to them, like the amount of data is new and frightening. That’s the big scary thing.”