A CONVERSATION WITH TOM REDMAN :

Data quality has been a continual challenge for almost every organization for many years. In this recap of a conversation with Dr. Thomas C. Redman, he offers a possible simple solution.

Anne Marie Smith (AMS): Subscribers to Data Management University should study “Only 3% of Companies Data Meet Basic Quality Standards” published September 2017 by Tadhg Nagle, Tom Redman, and Dave Sammon in Harvard Business Review. I caught up with Tom, a long-time friend and colleague, to discuss several aspects of this article.

AMS: Tom, can you describe the genesis of the “Friday Afternoon Method” (FAM) that is described in the article? Why should a team do it on a Friday, in the afternoon, and should it last two hours?

TCR: I’m a little surprised you asked about FAM first, Anne Marie. The headline is really critical here and we in the data management community need to face some hard realities. Everything we want to do depends on quality and the evidence suggests that data quality levels are horrifying! Perhaps those data governance programs simply aren’t working.

AMS: Well, then, Tom, let’s talk about the article. If only three percent of companies’ data meet basic quality standards, what do you think organizations should do about their data governance and data quality programs to improve them? In my experience, most organizations do not have best practice-based data governance programs at the enterprise level, so they cannot cause real, lasting change in the quality of their data. Do you agree?

TCR: I agree with you that most organizations don’t have best practice data management programs in place. And while all organizations are different, I think the issues are much deeper and more fundamental. Some don’t have the right measurements, some realize quality is bad but don’t realize the enormous costs, some view quality as a technical problem, and so forth. I don’t think  enough of us in the data community have done enough to really understand the issues, develop powerful business cases, and build the political support needed to take on data quality.

Now to your questions about FAM. I call it “The Friday Afternoon Method” because I noticed that people never seemed to have time to make a data quality measurement. Also, I noticed that a lot of people start thinking about the weekend about lunch time on Friday. So, I took up the challenge of providing a powerful procedure that could provide a solid first estimate of data quality and be accomplished by a data management professional in a Friday afternoon.

I haven’t kept hard statistics on this, but many people report that FAM works well any day of the week.

Sample Data Quality Report by Program (FAM)

Table 1: Sample Data Quality Report by Program

AMS: Your focus is data quality improvement, and your professional career has been focused on this field. So, are you surprised by the results that you have seen in the FAM exercises with your Ireland program students? Why or why not?

TCR: No, not surprised in the least. I spend most of my time advising companies and, let’s face it, they don’t call me in when their quality statistics are in the high 99s. I also talk to a lot of people. Most don’t think of data quality enough, but when I bring it up, almost everyone complains.

AMS: Do you see any correlation between the data quality issues that exist in companies and the lack of attention to other data management domains, such as metadata, or data architecture?

TCR: It is certainly true that inattention to metadata and architecture contribute to data quality issues. For both quality and architecture, I see the same sort of paradox: Organizations are simultaneously wasting too much and investing too little. Everyone knows quality is important—so much so that they spend enormous amounts of time fixing up the data so they can do their jobs. These are those hidden data factories you may have heard me talk about. What they don’t do is invest in preventing future errors. And, the madness—think about how crazy this is—continues!

Similarly for architecture. Today, organizations go to a lot of trouble to get their “systems to talk.” When you dig a little deeper, you find they never architected them to do so in the first place. In effect, they created an enormous problem by design. And, they continue to do so! I believe John Zachman said much the same thing here a few months ago.

These data issues (poor quality and architecture) are enormously expensive. And, they slow organizations down when it comes to anything else, like becoming data-driven or monetizing their data.

AMS: Would you recommend that an organization implement the FAM approach as part of its culture? If so, what would you recommend as the process it should follow to sustain such an activity?

TCR: I recommend that organizations implement a first-class data quality program tightly focused on their most important data. FAM can be an important part of that, particularly in getting started. FAM is unmatched at determining whether “you have a problem,” quickly and at low cost. But organizations have to go further, finding and eliminating root causes, putting solid controls in place, and so forth.

This is largely about getting the right people in the right roles. Each of us, everyday, is both a data creator and a data customer. And, if we step up to doing a better job at both, then quality improves quickly. None of the work is particularly hard, but it is unfamiliar. Hence the need for provocateurs and leadership.

Your question about culture is interesting. As I see it, culture is made by deeds not words. Once even a few people get in the habit of better articulating their needs, reaching out to customers, interpreting FAMs results, and similar efforts, the culture around them changes pretty quickly.

AMS: Thank you, Tom! Your insights, as always, are truly valuable and always worthwhile! DataManagementU.com appreciates the time you have given for this interview and we hope you return for another conversation.