Data-Driven Indecision Making

There is a strong degree of hype in organisations about digital transformation and data-driven decision making. My friend Doug Laney has written a fantastic book, Data Juice, about how organisations can tap value in data in the organisation. Between digital transformation, data driven decision making, and the desire to turn big data into gold, it’s a busy time in the data world. But it’s important to reflect on why we are doing all of this? What are the outcomes we are trying to achieve? And what data is key to those decisions and actions?

A personal vignette

As mortgage rates rose globally in the latter part of 2022, I contacted my lender in December to ask about fixing the rate for a few years. I was also looking at using some of the equity that has built up in my home over the past few years to finance some improvement works (the kitchen is showing it’s ‘here since we bought the place’ vintage). Being a prudent data-driven decision maker I wanted to get a reliable planning number for the family budget for the next few years. So the finance for home improvements would come AFTER we’d locked in our mortgage for the next few years.

The expected process outcome was that I would receive a letter from the bank a week or so after I asked for it that would tell me the new repayment amounts at different terms of fixed interest. And then, early in this year (2023) I could start looking at the home improvements budgeting and planning.

The expected information outcome was that I would have information about how much the repayments on the mortgage would be for the next few years and what the interest rate would be. Section 129 of the Consumer Credit Protection Act 1995 would seem to suggest that that is the minimum that should be provided. For me, it is key data I need to enable data-driven decision making on the next steps in this process.

What happened next?

By late January I hadn’t received a letter. I contacted the lender and raised a query on this. A few days later a letter issued. But as interest rates had gone up, it was no longer for the rate I would have been entitled to in December. This raises an important point about the timeliness of data in data-driven decision making. Some data has a short ‘shelf-life’ and needs to be available quickly to allow decisions to be taken. Other data can be aged like a fine cheese and still be good enough to consume, but as interest rates had gone up twice between me requesting the letter about my fixed rate options and me actually getting the letter, this data caused some stomach upset.

Root Cause: A process error in the Mortgage Operations team meant my request in December 2022 wasn’t acted on. They were only able to issue letters now with the current interest rates on them.

I contacted the lender again about this problem. To paraphrase Obi Wan Kenobi: “those weren’t the rates I was looking for”. What I wanted was the rate they should have told me about had they not screwed up the process outcome, which resulted in a failure to deliver the required information outcome, and impacting my data-driven decisions.

What happened after?

After a meeting with the lender where I handed them a list of interest rates that I had found on Archive.org I magically received a fixed rate letter the next day which had, co-incidentally, the exact same interest rate as I had pointed out as a possible interest rate that I should have been offered in December.

Note: I provided the data. I do not know if the interest rate that they have now offered me is actually the interest rate that would have applied in December. One would expect a lender to have some historic data on that kind of thing.

However… a key piece of data was missing from the letter. The amount of the monthly repayment that would apply. Also missing was any information on the indicative cost per thousand euro of borrowing. More accurately, that information was not provided for the interest rate that they were offering me (but it was provided for a lower rate. Curses… maybe they did know something I don’t know).

Remember, for my data-driven decision a key piece of (legally required) information is the amount of the monthly repayment over the fixed rate period. Also, the letter to accept the new terms on our mortgage required me to sign that I had been given this information, which I haven’t been given.

So I contacted the complaints department again (multiple phone calls and voice mail messages to little avail). And today, almost three months after my initial complaint and over four months after my initial request, I got another letter from the lender.

What’s happened now?

Today I was informed, in writing, that because they don’t have an automated process to calculate the repayment amount or provide the indicative cost per thousand euros and they are not able to calculate this manually, they won’t be giving me the information. The information that they would have had available in December if they’d acted on my request when it was made.

The information they are legally required to provide to me and which I am required to sign an agreement saying that I have been provided.

The Lesson

I suspect what has happened here is an instance of O Brien’s 5th Law of Data Transformation.

If you make the magic box a black-box process, you won’t know if the cat in that box is alive or dead. In other words: when you have to work outside the box in an emergency you might not know how.

In short: because my lender has developed their IT systems with a ‘forward looking’ happy-path focus, they are not able to handle it when something happens that is outside the process. Like a customer wanting them to give him the interest rate he was entitled to before the process failed, and who actually reads the fine print on the things they send him to sign.

This is the risk of over-optimisation and over-automation without investing in the documentation and codification of knowledge of how things are supposed to work. It’s the same enshittening of knowledge that Plato wrote about in the Phaedrus, and it’s the same enshittening of knowledge I wrote about here back in January.

And it gets tricky when the required information outcome (telling me what the monthly repayment will be on the remainder of the mortgage over a 5 year period at a particular interest rate) is a legal requirement.

Question: What processes exist in your organisation that are at risk of over-automation or being digitally transformed into a potential liability?