Every data documentation effort should identify the expected benefits, often based on the context and content
While defining an approach to documenting data and the relevant attributes to be documented, it is important to focus on depending on the context and the expected benefits of a data documentation approach. Data documentation is an often-overlooked activity in enterprise data management, to the detriment of the organization’s ability to manage data and operations effectively.
Documenting data is as much a necessity as an obligation. For any organization, the expected benefits can be expressed in terms of:
- Individual or collective productivity
- Definitions and other information related to a given data (business rules and other metadata) are documented, time is saved in the search (productivity in the search for information);
- Definitions and other information related to a given data (business rules, responsibilities, etc.) are documented, requests and other operations requiring these data (decision making) are processed more quickly (more reactive);
- Processing stages are documented, time is saved in the implementation of controls (control effectiveness);
- Processing stages are documented, time is saved in impact analysis (impact analysis effectiveness);
- Errors are reduced – higher quality data is available, money is saved from error correcting (reduction of non-quality costs and more generally operational support costs);
- Errors are reduced, data is delivered in time and in the right place (productivity in the data delivery);
- Less time is spent correcting errors, time is saved for other tasks (user devotes more time to his main activity);
- Staff share the same data documentation; they understand each other better and lose less time getting together (fluid collaboration);
- Systems are sharing the same metadata repositories, data exchanges are easier between systems, organizational units (regions, business units, etc.);
- Etc.
- Usability
- Data definitions, rules and local requirements are documented, interpretability and comparability are improved (fluidity of cognitive processes);
- Data definitions, rules and local requirements are documented, transposition, reconciliation and consolidation are improved (fluidity of upstream analytical processes);
- Data rules, controls and quality levels are documented, downstream analytical processes and data usages can be clarified and justified;
- Etc.
- Risk and cost control
- Data authorizations, restrictions and prohibitions are documented, then data are used wisely;
- Risks associated with the data are documented, controls can therefore be implemented in the right place;
- Reliable impact assessments are carried out, so the projects’ failures are limited;
- Data definitions and related information are documented, the effects of staff turnover are controlled;
- Etc.
- Regulatory compliance
- Data definitions and related information are documented and can be expected to reflect regulatory requirements;
- Processing stages are documented, results can be justified;
- Processing stages are documented, results can be replayed;
- Processing stages are documented, checks done, and related information can be proven;
- Etc.
The effort required for beneficial data documentation is not negligible and must be oriented by data classification. Indeed, the decision to document the data must prioritize critical data (whose unavailability penalizes the execution of a process), sensitive data (whose diffusion is likely to affect data subjects’ rights and freedoms) and regulatory data (a consideration for those subject to the regulations). For all these:
- Data quality, protection or security rules must be stated and documented;
- Data lineage must be developed;
- Data quality should be measured;
- Data quality issues must be managed and resolved;
To adopt such a strategy, an organization must develop three types of capabilities:
- Organizational, to define (policy, stewardship model, governance bodies, roles and responsibility matrix), execute and control (evaluation, decision, arbitration, resolution, audit) the metadata processing;
- Functional, in terms of metadata modeling (enterprise, conceptual, logical, physical and operational), processing (acquisition, storage, retrieval, collaboration, administration, distribution) and delivery (dictionary, catalog, lineage, reporting);
- Technological, in terms of metadata architecture, software and infrastructure, to enable users (business, technical or operational) of the documentation to quickly find the data meaning, the data rules and the associated levels of quality, protection or security; consuming applications, producing applications, golden sources; the data processing steps, from creation to use through the transformations undergone; the roles involved in the processing steps, the responsibilities in defining the data and associated rules; etc.
Conclusion
Documenting data is as much an opportunity as an obligation. The data concerned may be different in both cases: accounting and risk data in the latter case and legal, commercial, and operational data in the first case.
