Affiliated with:

Business Oversight Role in Data Warehouse and Analytics Solutions

Business Oversight Role in Data Warehouse and Analytics Solutions

Every organization needs a plan for business oversight of all data warehouse / business intelligence / analytics efforts.  There are certain areas that should be the focus of this business oversight.

Data warehouses, data marts and all analytical solutions are considered to be technical structures that are defined, built, and managed by the IT department.  However, the solutions must be business driven, having a real business problem to solve, and IT needs to have some documented requirements of what the business wants to do with the data.  While IT needs to be responsible for the architecture, applications, and technologies used to populate and deliver these solutions, the business needs to fulfill a crucial role in oversight of the content and usage of any data warehouse /analytics solution.

This view of solutions governance is focused on the management of the data going into the solution, who is able to access it, and is actually using it, as well as how it is being used.  It is imperative to understand the various roles and functions, and it is important to examine the crucial aspects of business involvement required to have a solution that realizes value and provides return on investment. A program to manage data governance at an enterprise level is essential for solutions governance success.

Content Oversight

Business resources / data stewards / subject matter experts are the actual sources behind the content of data in their capture systems and thus should be the ones providing the oversight of the data population.  The business participate in identifying the data and metadata used to populate analytical solutions, as well as the management of data and metadata as it is updated for the solution.  The key areas to map for business involvement include:

Data Mapping and Profiling:  Data in sources regularly changes in both content and quality.  This could be due to new source system changes, or business process changes that cause data capture or content to be altered.  While this is thought of as only a project assignment, it is also an activity that needs to be maintained and frequently validated. 

In both cases, this activity should be a joint effort between the business and IT resources that have to manage the tools, technologies, and database.  Establishing the processes and frameworks that support this will enable a repeatable structure that can be used across all analytics efforts in an organization. This area provides an opportunity to work with the sponsors, owners, and custodians of the source systems to educate them on the purpose and design of the analytical system.  Where required, the organization may have data usage agreements and security processes that help those key resources understand the level of protection and auditing provided for any data integrated into the solution.

Metadata Content:  There are several forms of metadata that are valuable to analytical solutions.  From business terminologies/definitions to data heritage/lineage, to operational metadata, all 3 areas need to be controlled.  Some of this data will be generated automatically and thus needs to be validated as new metadata is available.  Other metadata is updated and maintained by the business.  Typically, IT provides an interface for the business to maintain this information, but the business also has a responsibility to publish these changes to users so the new information is shared proactively.  Users tend to rely on metadata to navigate solutions.  Therefore, as users become comfortable understanding the context of data, any metadata changes should be communicated prior to making them available to users so they understand resulting change.  Connecting these changes directly to the analytical applications so users can be reminded of them upon login can help to cement the bond between the user and the content.

Data Acceptance:  While this seems to be a simple validating and rubber stamp certification of data, it is complicated by the fact that most data warehouses are loaded with initial one-time loads of historical data as well as recurring periodic updates.  Building on the process the organization has established for the mapping and profiling of data, it is essential to have a process for data acceptance.  Expanding the business and IT relationships can offer many benefits: results are managed, tracked, and made available to the business and IT leadership overseeing the solution.  Typically, most of the same resources are included to maintain consistency in knowledge of the data, mapping, and quality.

Metadata Acceptance:  Metadata is the secret sauce in describing the data and solution to users of analytical systems.  Having the context around the data they are using enables them to adopt and leverage the resulting solution.  Metadata acceptance is, in theory, similar to data acceptance.  It can be handled by following the same processes and tools developed for data acceptance.  The business and IT resources may be the same or, depending on the size and complexity of the solution, may be separate areas.  The only real difference is that the organization can address the various components of metadata that are attached to the data and resulting front-end solution.

Data Quality Assurance:  Once the solution is active, data quality becomes even more important.  As users grow comfortable with their analytics and tools, any data changes that affect the meaning and derivatives / metrics can have a strong negative impact on users if not understood.  Leveraging the same resources (both IT and business) and processes that conduct data acceptance can enhance that process to ensure the continuing quality of the data in a solution.  Publishing data quality issues will help users avoid challenges or concerns that require significant research on their end only to find out the data in the database has changed.  One best practice is to have business and IT resources responsible for managing the quality of analytic data sit on the committees that oversee all source system releases.  Knowing in advance about future changes will allow the teams to be better prepared and when to schedule special data quality audits.  For large or significant changes, this can help facilitate getting the analytics resources included in system releases so they can see the data as it goes through requirements and system/integration testing.  Once a system is in production, this joint collaboration enables business resources that best understand the data be the first line of support for data-related questions from the users.  The more technical and complex the data, the more valuable this support structure is.  In healthcare, the users of these systems are often providers of care.  Having a care provider talk to an IT resource, commonly an ETL developer, when they have a data question often results in frustration, misinterpretation, and lots of extra effort in fixing data that may just be interpreted or displayed incorrectly.

Organizational Alignment:  Depending on the size of the organization, there are many ways to arrange the various resources and teams in support of data content.  One best practice is to connect the business and IT resources involved in content oversight.  This helps to create a shared vision, frequent interaction, and joint ownership of results.  All resources do not have to report to one manager/director.  While one organization may report to IT and the other to a specific business area, it is important that there are dotted line relationships, joint meetings, and joint leadership interaction/supervision.

Conclusion

Every organization must have a plan for business oversight of all data warehouse / business intelligence solutions.  More importantly, the organization must be prepared to execute this plan and revise it when necessary.

Share on linkedin
LinkedIn
Share on facebook
Facebook
Share on twitter
Twitter

Bruce D. Johnson

Bruce D. Johnson is an experienced IT consultant focused on data / application architecture, and IT management, mostly relating to Data Warehousing. His work spans the industries of healthcare, finance, travel, transportation, and retailing. Bruce has successfully engaged business leadership in understanding the value of enterprise data management and establishing the backing and funding to build enterprise data architecture programs for large organizations. He has taught classes to business and IT resources and speaks at conferences on a variety of data management, data architecture, and data warehousing topics.

© Since 1997 to the present – Enterprise Warehousing Solutions, Inc. (EWSolutions). All Rights Reserved

Subscribe To DMU

Be the first to hear about articles, tips, and opportunities for improving your data management career.