Data Quality – a bottom line cost for society
Data Quality is often considered to be an abstract topic. After all, unlike a meal in a restaurant or a piece of furniture, the quality of data is often difficult to appreciate in a tangible way. And, quite often, the root cause of a data quality problem can be a simple error that compounds over time.
Making it real
The IMI and UCC published research back in 2017 that found that less than 3% of organisations have data that meets basic data quality requirements (yawn). The average cost of poor quality data in an average organisation is consistently estimated as being between 10% and 30% of turnover (yawn). Statistics like this appear not to register with decision makers in organisations who continue to embrace the lure of the next shiny technology as a way of milking greater value and productivity from their data assets.
Perhaps what we need to do is to make it real. Which is why a number of years ago I started a blog called IQTrainwrecks.info to document some of these examples from the media of things that have happened that have a data quality root cause. That website has been sadly neglected in recent years as I had taken to posting things to Twitter with the hashtag #IQTrainwreck instead.
But having stories can help make the abstract tangible. Having a tale of woe to illustrate a point can help it resonate and make people think.
Getting to the punchline (or the bottom line)
Every good story has a punchline, a payoff. And the more tangible or visceral to the reader the story can be made the better it is at getting the point across. So, it was interesting to learn in this time of rising energy costs and other inflationary pressures that, for over a decade, a data quality issue has resulted in Irish consumers paying more in their electricity bills to subsidise large energy consumers like data centres.
The statement from the Energy Regulator describes this as an “administrative error” in the way the subsidy had been applied. The admin was wrong. The data was not as expected. The consumer was overcharged. Nobody seems to have noticed for a decade.
That’s some error.
That simple “administrative error” resulted in every single household in the country paying more than they should have for electricity for over a decade. The total cost of the subsidy was €600 million. It is unclear what the amount of the overcharging actually is.
What was the administrative error? It would appear that instead of the subsidy being levied as a fixed charge, which was the intention, it was in fact levied as a percentage. So, as the consumption by large energy consumers went up from 2011 to now, the cost to households went up in parallel.
Now, it’s important to remember that the period 2011 to 2023 didn’t have much happen that involved data centres, apart from the growth of online streaming services, online retail, cloud-based services, social networking…
To put it another way… households seem to have been paying for their Netflix twice. Once in their Netflix subscription and once again in a hidden charge on their electricity bill.
How much has this cost households? We do not yet know, but estimates have floated at least €100 per household. But that assumes households have been static since 2011. It’s not like people have moved house or things like that in the meantime.
The Moral of the Tale
A data quality problem implementing the wrong calculation mechanism, combined it would seem with a failure of oversight and data governance has resulted in:
- At least €100 per household of overcharging
- A complex data puzzle to find out who was affected, how much they are owed, and how to repay them the money that was taken.
And all an avoidable cost of non-quality.