IAIDQ Blog Carnival

Submitted by Daragh O Brien on February 14, 2011 - 13:16

So, it is that time of the month again when we look back at the posts which educated and entertained us from the various data quality bloggers who are part of a community that has grown rapidly in the nearly 3 years the IAIDQ has been running the Blog Carnival.

First up is Kasper Sørensen, the founder of Data Cleaner, an open source data quality tool. Kasper kindly shares with us an insight into one of the common conversations that organisations inevitably have when it comes to tackling Data Quality Problems - the "hand carve it from solid SQL" or "bring in a specialist tool". I know I've had this conversation myself many times.

Check out Kasper's conversation on his blog here.  

Next we have the irrepressible Jim Harris of OCDQBlog.com who (amongst other things) got philosophical with us in January when he discussed the Asymptote of Data Quality. This is an important concept that often escapes the mindset of technologists (who want a definitive answer) and business-side people (who often want an easy answer), or project-driven people (who want an answer that can be time-boxed and packaged). Over time the cost to achieve perfection in data can far exceed the benefit of "good enough". Perfection might be a target, but there are economic factors to consider. Often it might be a better use of resources to "hold the gains". (I wrote about something similar back in 2008 for the IAIDQ).

Finally, Dylan Jones from DataQualityPro submitted an interview with Martin Humphries from Essent who has successfully appliedLean techniques to Data Quality, which also gives an interesting counter point to Kasper's discussion re: SQL vs Tools.

 

So, what's the theme that links everything in my view?

 

  • There are many paths to Data Quality excellence.
  • Organisations need to accept and understand that perfection isn't always possible, valuable, or necessary (but is a good target)
  • When it comes to tools vs SQL and the costs/benefits on each side there are trade offs which need to be considered (personally I prefer tools based approaches but SQL is always an important skill set).