Building a Data Marketplace is not an easy task, requiring innovative, disruptive, and differential technical and cultural solutions. Follow some useful tips on organizational, cultural, and technical change management that support data marketplaces
How to build an internal Data Marketplace and make the most of it in an organization?
An effective internal Data Marketplace enables the enterprise sharing of data and information across roles, levels, technologies, and other common barriers to the optimal use of data as an essential asset.
Starting from technology, innovative, disruptive, and differential technological solutions may offer not only the appropriate features and functionalities but also a series of value-added capabilities that allow the organization to make the most of this new paradigm.
It is essential to find the right technologies designed to help organizations with the effective and efficient operationalization of their data strategy thanks to the implementation of proactive and preventive data governance with a focus on extended metadata management and automation of common technical processes.
One of the most important aspects of proactive and preventive Data Governance consists of anticipating needs and making processes more efficient, based on business requirements, which then must be implemented on the technical layer. To implement this new approach effectively and obtain the maximum possible value, one of the keys lies in the automation of common technical processes through the activation of metadata. This requires a bidirectional integration between the different layers of the Marketplace (Governance Layer, Security Layer, Data Layer). The concept of DataOps supports this approach.
Starting from the most basic requirements of data governance, the involved technologies must allow an organization to gradually evolve to cover the different requirements of the various stakeholders of a Data Marketplace, and thus become the backbone of an organization’s data ecosystem.
Of course, technology alone is not the solution. Some tips and useful thoughts may ease the implementation of a successful Data Marketplace:
- First, think about maximizing business results, not about the underlying technical processes or only about regulatory compliance (e.g., profit increase, cost reduction, efficiency increase, …).
- Define a collaborative model for data governance based on roles and data domains, with clear responsibilities and functions for all stakeholders (e.g., data stewards for business areas, data architects as enterprise roles, data engineers for technical asset management, …).
- Involve the participants and stakeholders of the data governance model as soon as possible and involve them in the change, looking for strategic allies.
- Define a metamodel that brings together all the knowledge and context of the organization’s data assets from both business and technical points of view according to current needs. Do not neglect future needs (e.g., reports, metrics, concepts and quality rules for the Business Glossary and repositories, datasets, and processes for the Data Catalog).
- Define agile and flexible policies and procedures that can be easily adapted and evolve over time as needed (e.g., data contract management policy, data access request procedure, dataset technical metadata change procedure, …).
- Define processes for data lifecycle management along with metadata enrichment and activation from a centralized metadata repository (e.g., sandbox generation processes for advanced analytics are configured based on metadata from the centralized repository obtained from assets stored in the Data Lake and available data contracts).
- Promote metadata enrichment by data producers and data providers as well as feedback regarding their understanding, comprehension, potential uses, and new needs from consumers.
- Develop and implement bi-directional integrations between the Data Marketplace Governance layer and data platforms to improve the management of data assets from a governance perspective agnostic to data capture, storage, transformation, processing, and exploitation technologies (e.g., automation in metadata extraction, active governance for data access permissions management, audit log monitoring, …).
- Design a specific communication plan and facilitate change management.
- Start with the implementation for one use case and then scale to other data domains, systems, or projects.
- Separate new production from stock from the data point of view and apply the governance-first or governance-by-design concepts for all new requirements, uses and projects.
- Provide and enhance employee training in Data Management practice, Data Literacy, and analytical capabilities.
- Build a one-stop shop for data integrated with demand management that facilitates the day-to-day operations of all stakeholders.
- Promote proactive and preventive data governance with a 360° view and best practices in the field of data management.
- Define taxonomies for data assets to help with data management (e.g., semantics and ontology, information sensitivity, applicable regulation, …) and promote their use.
- Relate business assets to each other and to technical assets, creating semantic maps and knowledge graphs to facilitate their understanding.
- Identify high-use and critical data and work on improving its quality and on generating logical groupings of data to facilitate its sharing in a regulated way.
- Promote among data producers and providers the creation of reusable logical groupings of data regulated by standardized contracts to make them available to potential consumers.
- Involve legal areas to ensure compliance with data contracts and the inclusion of specific clauses for both producers and providers as well as consumers.
- Work on the creation of data valuation algorithms based on their criticality; their acquisition, maintenance, and availability cost; their quality; their different uses; and the risk that their storage and use represent for the organization.
- Create dashboards, metrics, and alerts for monitoring the Data Marketplace and its processes with a focus on continuous improvement.
- Incorporate to your technological ecosystem differential, scalable and interoperable solutions with multi-platform and multi-cloud vision and that offer the necessary flexibility and adaptability to face current and future challenges with guarantees.
Conclusion
Successfully building an internal Data Marketplace for an organization is a major challenge for people, processes, and change management. However, it also may bring many benefits that can make a difference for achieving the goals set by the data strategy.
Finally, the incorporation of new technology is not the main objective in this case, but it must serve as a support for whole process of change and evolution of the data paradigm of an organization. Having the right technology will can help achieve the established goals to implement an internal Data Marketplace.
