The Data Transformation Renaissance: Are we building castles on sand?

There is a resurgence in focus on data transformation as we adapt to a post-Covid world. This renaissance is fuelled in part by a year’s worth of mothballed projects. But it is also driven by a realisation among organisations of all sizes and across all sectors that they are, first and foremost, in the business of data. Triggers include the transformation of traditionally office-bound jobs to connected working and the shift of historically ‘bricks and mortar’ businesses to online or hybrid models.  But we can’t discount the simple impact of management and staff realising that getting simple answers to critical questions in their organisations was not as easy when their workforce was distributed. The cracks in information flows that are all too often papered over by access to co-workers became a lot more apparent.

On Twitter earlier this month, the CIO of BOX wrote that a dam had broken in 2020 and we were now we were seeing a tidal wave.

We’re about to have a renaissance in enterprise software and IT. Every CIO I’ve chatted with in the past couple months is embarking on significant digital transformation right now. Last year the dam broke, and now it’s a tidal wave.

— Aaron Levie (@levie) June 8, 2021

Sand Castles against the Tide

Ignoring for one moment the crime against metaphors that has been committed, it’s worth reflecting on Mr. Levie’s statement and what it means for organisations of all kinds. For a start, he refers specifically to “enterprise software” and “IT”. This is understandable. But it should be seen as a parrot, dressed as a canary, who is standing in a coal mine and has just coughed and opined that it is pining for the fjiords. (OK, metaphors can be hard).

This comment invites the age old mistake of conflating information and data with the technology that is used to manage and process it. And that mistake has lead countless organisations over the past 30 years of data-enabled business transformation into the environment where the failure rate of data transformation projects remains stubbornly unchanged at between 60% and 80%.

I actually wrote about this topic back in May of last year, just as we were settling into the first of (many) lockdowns. Unless in this renaissance we learn from the past failures of data transformation, we are building our castles on sand and, when the tidal wave passes, those castles may be washed away.

The Data Transformation Renaissance: Strategy

The renaissance in enterprise software and IT will not be sustainable, and certainly won’t be achievable for smaller organisations, if there is not a renaissance in data strategy first. I say renaissance, but I actually mean a recognition of. Fundamentally, when a technology investment is being considered, the organisation has to invest some time in ensuring a clear strategic plan for data to support their business. And that needs to be tied to the formulation of a business strategy.

• What do you want to do as a business?
• What are the business capabilities you will need to have to achieve that?
• What data capabilities will you need to have to do that ? (People skills, process improvement, technology platforms need to be considered equally)
• How will you get those things?

All too often, however, organisations get tunnel vision on “technology as transformation” and focus on swapping in or out technologies to address a perceived problem or gap in their business. Castlebridge has worked with a number of clients who were about to fall into the trap of investing in technologies to improve their businesses without an appreciation of the “data” part of the data transformation. By helping them look strategically at the data component of their planned transformation, we enabled those clients to understand the data capabilities that would be needed to deliver a sustainable and successful transformation. [Case Study]

The Data Transformation Renaissance: Data Quality

It should be really hard to ignore the 60% to 80% failure rates for data transformation projects. But unfortunately organisations all too often do. Which means they fail to plan for the inevitable quality problems that lurk in their data, often masked by workarounds or ‘local’ knowledge. According to UCC/IMI, less than 3% of organisations have data that meets basic data quality standards. That costs organisations between 10% and 30% of turnover annually on average. If your data transformation initiative is not working to improve the quality of the most important information for your most important customers (internally or externally), it will more than likely fail.

Data quality is not addressed by spending money on data cleansing. That’s throwing good money after bad and doesn’t actually add any value. Data Quality requires you to understand the data needs of the various processes in your organisation and ensure that you have designed and implemented your ‘data factory’ in a sustainable and scalable manner. So, to improve data quality you may need to look at the data model for your organisation, the business data glossary, or your metrics and incentives for data capture or data processing. Castlebridge has recently completed an engagement for a client where we were asked to “help fix some data quality problems”. Our analysis found an organisation whose underlying data model (which had been defined early in the organisation’s life) didn’t match the realities of the organisation today and couldn’t possibly provide a foundation for the organisation model they wanted to move to in the future. No amount of data transformation investment that didn’t tackle the fundamentals of quality in the specification, modelling, and capture of data would deliver a benefit to the client. We have mapped out a roadmap for improvement starting with some fundamental foundations, like a Business Data Model and basic data quality standards and metrics.

The Data Transformation Renaissance: Data Governance

Data Governance deals with the definition of how people are expected to behave with respect to information-related processes. Data Governance is essential to the sustainability of any data transformation as it maintains alignment between implementation and strategy, and ensures appropriate oversight and monitoring of quality. It also has a key role to play in compliance with data protection legislation, management of information related risks, and ensuring alignment between the various data management disciplines that might be contributing to your data transformation.

Ensuring alignment between People, Information, and Technology is a key aspect of Data Governance. And at the heart of a good Data Governance programme is effective change management that ensures that everyone knows their responsibilities and accountabilities in relation to data and information. If we miss this foundation, the sustainability of our data transformation will inevitably be in doubt. Initiatives that are not aligned with data strategy will be initiated, changes will be implemented without appropriate controls, and the discipline and rigour that is needed to deliver the transformation will not be sustained in the organisation.

Castlebridge has worked with a range of organisations helping them to define Data Governance roadmaps that take into account a wide range of data strategy objectives, data quality pain points, and compliance obligations relating to data and its use in the organisation. It is essential that organisations trying to build a sustainable data transformation invest just as much in the People and Processes for that transformation as they do in the technology. [Case Study]

The Data Transformation Renaissance: Regulatory Compliance

It doesn’t matter how good your data transformation is if it is just helping you do things in a manner that could be shut down by a regulator. For most organisations, the main regulatory focus will be on data protection compliance, but other laws and regulations may be relevant. The pace of change world-wide in this area means it’s important to identify and plan for regulatory or legislative requirements that will impact on your data plans.

From Connected Working to implementation of Machine Learning and process automation, it’s imperative that your Data Transformation consider how you will comply with or meet regulatory requirements. These are not “nice to have” but are essential. Compliance with the law, after all, should be the floor not the ceiling of our ambitions in what we do with data. Castlebridge has worked with clients across a range of industries and a range of regulatory regimes (data protection, utilities regulation, healthcare) to help ensure an alignment between the Data Strategy of the organisation and the Data Regulation that it must comply with. Often this involves translating regulatory requirements into a clear plan for reporting and disclosure of information or the definition of operational and technical controls to ensure that data is managed correctly in the organisation.

The Data Transformation Renaissance: Ethical Data Management

The final foundation pillar of our Data Transformation renaissance needs to be Ethical Data Management. As ESG continues to grow in importance for organisations the ethical aspects of data management cannot be ignored when defining our data strategies and mapping out our data transformations. Investors are increasingly looking to organisations to demonstrate how they are meeting their Environmental, Social, and Corporate Governance obligations. That requires data. And that needs to be part of the data strategy of the organisation, underpinned by good data governance. And it’s often driven, or at least triggered, by regulatory requirements.

The Castlebridge team quite literally wrote the book on Ethical Data and Information Management.

Conclusion

One hallmark of the classical Renaissance of the 15th Century was the rediscovery of wisdom and knowledge that had been lost or forgotten during the early medieval Dark Ages. In our Data Transformation Renaissance we need to relearn and learn from the lessons of our data management past.

Otherwise, we will make the same mistakes, projects will underdeliver or fail outright, and this golden opportunity to deliver sustainable change for our organisations, our teams, our customers, and our society as a whole, will be squandered.

Let’s not build our castles on sand (this time).