I’ve been reading a lot of interesting blog posts in a variety of places about the importance and value of metrics for data quality and the potential for misunderstood measurements to drive misunderstood (or just plain wrong) decisions.
At the simplest level, this is yet another iteration of the age old “Carpenter’s Rule” – Measure twice, cut once.
But carpenters have it easy. Irrespective of your level of experience in carpentry and woodwork you intuitively know that the measures that matter to you are length, width/breadth, and height. All of those are ultimately different dimensions of the same metric (width is length from a different perspective after all). The key challenge for the carpenter is to make sure that they are measuring in the same units of measure (inches, feet, metres, centimetres). And if they are working with other carpenters they need to make sure that they have agreement on what unit of measure they are using.
I’ve seen the fun and games that ensued when my father (an imperial measurement thinker, living in France) was trying to work on a project with his neighbour (a Frenchman who just did metric). Assumptions (on my father’s side) about what the “20” meant on a plan diagram were inevitably going to be wrong if he didn’t stop and remind himself he was in the land of metric measures. (If only NASA had done the same checks with the Mars Orbiter space probe which crashed because different teams were using different units of measure during coding of the telemetry systems).
But is length the only “measure that matters” to a carpenter? What about how straight (or curved) things are? Marking up to cut a non-straight line when you need straight line cuts leads to waste. But there are instances where curves, angles and crooked lines are required and have to be produced consistently. Being able to properly set up a rig to cut the correct angle is an important skill. What is the unit of measure? Is it an angle, measured in degrees (for example – if you are cutting a mitre on skirting boards?) Is it an arc (defined and measured by a series of angles) ? What about the weight of what will be on the shelves? Should that be a factor to measure and consider?
And what about precision? Is the level of accuracy that is required the same for rough and ready shelving in your garage as you’d expect in fine hand crafted furniture?
Ultimtately it is down to the carpenter understanding what the strategic goal is… what the customer needs. That affects the choice of key metrics, the level of precision that is required.
In my video tutorial about the relationship between Information Quality Management and Data Protection I talk a little about the different “Dimensions of Data Quality” that exist. It is important that organisations (and information quality professionals) remember that at different times in the life cycle of information different metrics may be more important than others. Also, for different processes and objectives different metrics will be more important than others.
David Loshin wrote recently about the pros and cons of the “Data Quality Scorecard” and I have to agree with a lot of what he has said. The scorecard is a good starting point to get attention, but only if you have measures in it that matter to people and only if you can translate “Percentage of Field X that is Null” into an actionable bottom line impact (e.g. – “we can’t rely on AML controls without that data – you’re going to jail boss”).
In addition, a single scorecard often isn’t going to cut it (no pun intended) as there are more stakeholders than just the exec team. The Information Quality team or Process Compliance/Governance teams may need more granular measures at different stages in the process life cycle. Different data may be more critical in certain processes steps than others. Different regulatory considerations may affect the when, where, why and how of capturing certain data.
So, just as a carpenter will make a number of different measures before cutting wood and doing assembly, so to the organisation will need to take different measures at different times. Ultimately all I care about in shelving that I am getting put up is that it will be sturdy enough to hold all my stuff. That is my top-line “Carpentry Quality Metric”. How that is arrived at is not necessarily my focus, but if that work in progress measurement hasn’t been done then, no matter how good the shelving looks it is probably not going to stand up for long.