The debate continues between centralized data governance initiatives versus federated or functional business unit approach, and between business-led versus IT-lead data governance programs.
Why do so many enterprise data governance initiatives fail? Data governance has consistently been recognized as a critical enabler of business success – yet it remains a massive challenge for most companies – with some failing outright to drive any meaningful value whatsoever from their data governance programs.
A recent survey from Gartner found that only 18% of business-led data governance programs claimed to be effective at generating business value for the organization. By comparison, the same survey found that 66% of IT-led data governance initiatives claim to be effective at driving business value. [1]
The difference between IT and business perspectives on the effectiveness of data governance is stark – and problematic. While IT leaders may see more value in data governance, a continued belief by business stakeholders in the inability of data governance to provide value will threaten the long-term viability of any enterprise-wide data governance initiative. This is because business engagement in data governance programs is simply not optional.
So why does this split between business and IT views of data governance effectiveness exist, and why do so many companies continue to see so little value from data governance programs?
An ongoing failure of companies to make a quantifiable connection between an investment in data governance and the delivery of business value is the biggest reason for data governance failure. If companies can’t prove that data governance yields business benefit, they will be less likely to continue to invest in it. In lieu of any form of forecasted ROI on data governance, it ultimately becomes either an act of faith, or a regulatory requirement. However, the problem goes deeper than a failure to build a data governance business case and model its benefits. The split between IT and business user perspectives of the benefits of data governance is a strong indictor of one of the less-discussed root causes of data governance failure, known as the Data Governance Cost/Benefit Tradeoff (see figure 1).
The Data Governance Cost/Benefit tradeoff highlights that the costs of implementing and managing a data governance program are skewed towards functional business units, while the benefits tend to naturally accrue at higher, more cross-functional levels of the organization. This is exactly what is highlighted in the Gartner data showing more favorable perspectives of data governance within IT – which is an inherently cross-functional endeavor.
Modelling the benefits of data governance is discussed often and is widely known as an industry best practice. However, this in only half of the equation. The costs most also be modelled and understood at every level of the organization participating in a data governance program. There is a tradeoff between costs and benefits, and for companies to succeed at data governance, a balance between these costs and benefits must be found.
A disconnect between the costs and benefits of enterprise data governance initiatives also explains why companies are on a constant pendulum of swinging back and forth between data governance programs with enterprise-wide scope, and those with more business-unit focus. The same is true with almost all their other data management initiatives beyond data governance, with a highly predictable pattern of yo-yoing between centralization and decentralization. Examples of such programs include metadata management, data quality, master and reference data management, etc.
Companies failing to realize cross-functional benefits of data governance will often turn to hiring a new CDO to solve the problem – or where a CIO is given a similar mandate to ‘fix’ data governance problems (often as a part of some other systems-centric initiative, like an ERP consolidation). The CDO or CIO, given the cross-functional nature of both roles, will then look to implement appropriate policies and supporting data architectures that best support cross-functional uses of data. The architectures supporting these policies tend to be more centralized, given the economies of scale these models generate.
This shift towards more cross-functional data governance policies with centralized data management architectures will produce a backlash in business units since the benefits (at a functional level) of defining, implementing, and managing governance policies optimized for cross-functional use cases will be questioned. Business functions will decry a lack of autonomy and business agility, which often can result in companies abandoning further attempts to implement ‘top down’ data governance, so functional units are ultimately given more freedom to manage their own data. (See figure 2)
In many ways, figure 2 is exactly what is happening right now with the increasing focus, and market hype, on data mesh architectures. A backlash against top-down, centralized styles of management and data governance is leading many across both IT and business groups to advocate for more federated approaches – where business or domain-level freedom (and benefit) is accentuated, but where the costs and complexities to support more cross-functional approaches to data management are pushed into IT.
This pendulum swing between hyper centralized and hyper federated approaches to managing data governance does not mean there are no benefits to decentralization. There most certainly are – and that’s why the data mesh is gaining popularity. The problem with highly federated data governance programs is the costs of data governance at a functional level are largely the same regardless of the program scope. Policies for data quality, access, retention, security, etc. etc. etc. must all be defined and managed – the only thing that changes under a federated model is the scope of those policies (local vs global).
However, the benefits of localized data governance outside the functional unit are negative given the burden placed on cross-functional coordination – since those local policies will often be inconsistent with more global requirements. Put more simply, how marketing defines a customer could be completely different to how finance defines a customer. Where there was a single set of data governance policies for customer data, there are now two or more definitions and policies.
In abandoning hub-and-spoke type architectures for what the data mesh calls ‘federated computational governance’, a significant number of point-to-point integrations must be created across domains and business units. This will drastically increase the cost and complexity of both data management and data governance for any data that needs to be shared at a cross-functional level. For many IT organizations, these costs will exceed the perceived benefit of local business unit autonomy. In the future, highly automated data management solutions may allow these cross-domain interactions to scale, but those solutions have yet to materialize.
Conclusion
The tradeoff between the costs of data governance, which includes governance policy definition, implementation, and management (including stewardship), and the benefits of governance, which tend to be greatest at more cross functional levels, explains why business units see less value from it.
In theory, the fix to this problem is easy: model the cost and benefit of data governance at every level of the organization. If an imbalance exists, where the benefits at a corporate or cross-functional level exceed those of the business units, the organization must find a way to compensate functional business units for their share of the costs of governance.
In practice, the answer here is much more complex – since an optimal steady-state is a more adaptive governance model that allows for both local freedom and cross functional benefits. Implementing truly adaptive governance models is possible, but requires high levels of organizational, financial, technical, and data management maturity that most companies lack. The first step to improving on these maturity levels is to find a way to measure both the costs and benefits of data governance at multiple levels of the organization. Until these metrics exist, the problem of data governance failure will persist.
[1] Gartner Inc, The State of Data and Analytics Governance: IT Leaders report Mission Accomplished, Business Leaders Disagree. Published 12/6/2021
