Insights

Data Literacy: The Zaphod Beebelbrox Approach to Managing and Mitigating Risk

By Daragh O Brien

September 17, 2020

22min read

In the Hitchhiker’s Guide to the Galaxy, Zaphod Beebelbrox (Galactic President, criminal, and all-round froody dude) has a unique approach to managing and mitigating risk. He has a pair of Joo Janta 200 Super-Chromatic Peril Sensitive Sunglasses. These glasses have a particular function. They help people maintain a relaxed approach to danger by turning completely dark and opaque at the first sign of trouble. In this way, the wearer is unaware of the impending danger, because they can’t see it.

The Problem

In many organisations, management dashboards have become the C-level equivalent of the Janta 200 Super-Chromatic Peril Sensitive Sunglasses. If you don’t understand the data that is going into them, and if the information is presented in ways that looks good (all the graphs that should trend up trending up etc.), you might not see the impending danger to the organisation. Depending on the lineage and context of data you might be making some serious strategic errors or working off some unsound assumptions.

Carl Bergstrom and Jevin West give some good examples of these types of issue in their book Calling Bullshit. Their website has some great case studies that highlight the need for people at all levels in organisations, and indeed in society, to improve their data literacy skills. Whether it’s evaluating claims of “99.9% caffeine free” or the ethical mine-field of some AI applications, Bergstrom and West’s work brings home the message that we really do need to up our game when it comes to this thing we call “data”.

My colleague Katherine’s blog post last week included a couple of other great examples.

This is essential if we want to up our game in managing and mitigating risk associated with data and its use or misuse.

The real problem

Of course, the problem isn’t the dashboards or the data. The problem is that people in organisations are the product of their education and experience. That education and experience might not have prepared people on either the “business” side or “technology” sides of the organisation for today’s data-centric world. And this goes beyond understanding how to use tools and technologies (although that is important).

It can sometimes mean having consider your biases towards data (is it an “IT thing”?). It definitely means recognising the gaps that might exist in your understanding of the different disciplines of data. Data Leaders don’t need to be experts in all things, but they do need to have some understanding of them. Only then can you really understand the risks you are managing and mitigating.

However, all to often, the leaders in organisations adopt another of Zaphod’s leadership maxims:

If there’s anything more important than my ego around, I want it caught and shot now.

The Impact

This JooJanta 200 approach to information-related risk can have impacts. Examples I’ve encountered over the years…

Organisations don’t measure data quality on an ongoing basis. Decisions are made on flawed data, leading to resource-intensive workarounds and rework. Even worse, investments are made in new systems or technologies to ‘magic away’ the data problems, but the problems persist.
Organisations don’t log “near misses” in data protection contexts (e.g. near miss data breaches). This means they don’t have actionable data about common causes of risk in their organisations.
Organisations don’t log changes to data over time (audit trail) meaning they don’t have the ability to verify how or why data was changed.
Organisations discount or ignore the cost of data debt and technical debt in projects, resulting in significant failure rates on projects.

Some Examples of Data Literacy and Managing and Mitigating Risk

Here are some examples of data literacy and risk management I’ve picked up over the years.

The Data Protection Metrics for Managing and Mitigating Risk

Imagine the scenario. I’m sitting in a meeting with Data Protection Officers (DPOs) from two public sector organisations. One DPO has a tracking report of data security breaches and reported ‘near misses’. The other does not. The DPO who lacks data criticises their peer for the number of breaches on their tracker. I ask what that DPO’s number is.”We don’t track that”, is the response.

This scenario happened to me over a decade ago. Just as not testing for an infectious disease means you don’t find cases, so your case numbers remain low, this DPO was blissfully unaware whether their organisation was leaking like a sieve (it was). The other DPO had Castlebridge in training their team on Root Cause Analysis techniques. They were learning from their near misses and improving controls.Which DPO was more likely to avoid disaster? Who was wearing their Joojanta 200 shades with pride?

Are you a DPO? Are you sniggering at that example, happy that you are measuring things like the numbers of Data Protection Impact Assessments that have been completed? What does that metric actually mean? What message is it giving your senior management about the performance of the organisation on data protection? Is this the data protection equivalent of winning at Scrabble by memorising the words (to borrow Katherine’s example)?

Managing and Mitigating Data Quality Impacts

Looking beyond data protection issues, your organisation is probably burning between 10% and 35% of turnover (or operating budget) coping with poor quality data. That’s the average impact, borne out by multiple studies over three decades. Over the years, Castlebridge has identified FTE staff costs in clients of between 10% and 15% arising from the reworking and ‘fixing’ of data. This is before we look at the costs arising from lost opportunities, compliance costs, and other factors. Recent research by UCC found that less than 3% of organisations have data that meets basic data quality standards. How can data quality levels like this help in managing and mitigating risk across your organisation? The answer is, they can’t. But we soldier on regardless.

If you want an example of data literacy and data quality and the impacts it can have, this story about a family falsely accused of distributing child pornography due to a misreading of a date is a good example.

The costs from poor data quality are often accepted as ‘the cost of doing business’. Organisations are flying blind and muddling through while data quality levels in the organisation are not measured and the costs of non-quality are not counted. They continue to absorb and accept direct costs of scrap and rework and the indirect costs of missed opportunities and poor returns on investment. That means leadership in the organisation need to understand data as a business asset and to understand how quality management methods can be applied to improving the accuracy and reliability of data. Without that knowledge, the significance of their dark Joojanta 200 lenses will struggle to filter through to the decision makers in organisations.

In Conclusion

As Deming said: “In God We Trust, everyone else must bring Data”. But it is important we develop the management competencies to properly understand, interpret, and apply that data. Without that contextual literacy, organisations will struggle to become information-enabled. Zaphod’s JooJanta 200 sunglasses are doubtless effective in preventing him from seeing danger ahead so he doesn’t get distressed. However, the very fact that they went dark was a warning that something was about to happen that could alarm him, if he could see it. And that warning in and of itself was potentially cause for alarm.

Some homework: Ask yourself what are the top 3 metrics in your organisation. Ask yourself how that data is gathered, what is its lineage and provenance. Then poke around the quality of that data. Not sure how to do that? Then you have found the first step on your data literacy journey.