Abstract:Data governance can effectively ensure that the data construction process is carried out under a reasonable and efficient supervision system, and ultimately provide high-quality, secure, and process-traceable business data.
The enterprise data governance system includes data quality management, metadata management, master data management, data asset management, data security, and data standards.
1. Data quality
Data quality is measured by standards commonly used in the industry: completeness, accuracy, consistency, and timeliness.
- Completeness: Whether the records and information of the data are complete and whether there are any missing cases
- Accuracy: Whether the information and data recorded in the data summary are accurate, and whether there are any anomalies or errors
- Consistency: The common data of multiple business data warehouses must be consistent across each data warehouse
- Timeliness: Data can be produced and warned in a timely manner
2. Metadata management
Metadata is information about the organization of data, data domains, and their relationships.
Metadata includes both technical and business metadata. It can help data analysts clearly understand what data the enterprise has, where it is stored, and how to extract, clean, and maintain such data, that is, data lineage.
- Help build a business knowledge system and establish the explainability of the business meaning of data
- Improve data integration and traceability capabilities, and maintain kinship relationships
- Establish a data quality audit system and classify management and monitoring
3. Master data management
Enterprise master data refers to the business entities that are consistent and shared within the enterprise, and in the vernacular it is the data shared between various professional companies and business systems.
Common master data such as company employees, customer data, organization information, supplier information, etc. This data is authoritative and global, and can be attributed to the company's corporate assets.
General master data management needs to comply with the following points:
- Manage and supervise the access to master data by various organizations, subsidiaries and departments, and formulate access specifications and management principles
- Conduct regular master data evaluations to determine the degree of completeness of the established goals
- Organize relevant personnel and institutions to uniformly improve the construction of master data
- Provide technical and business process support, centralized across the group
4. Data asset management
Generally, enterprises consider data asset grooming when they are in digital transformation. Is your data being used wisely, and how can you generate the most value? This is the core work of data asset management. When constructing enterprise assets, different perspectives are generally considered, that is, business perspectives and technical perspectives, and finally merged, a unified data asset analysis is output, and a unified data asset query service is provided to the outside world.
How to revitalize data, form data assets, and provide a complete panoramic view of data assets, which can facilitate operators to control the dynamics of enterprise assets globally and macroscopically.
5. Data security
Data security is an indispensable part of enterprise data construction, and our data is stored in large and small disks, providing different degrees of query and computing services to the outside world.
Data needs to be regularly audited, sensitive fields encrypted, and access rights controlled to ensure that data can be used securely.
6. Data standards
In the vernacular, we need to define a set of norms for data within the organization so that we can all understand what that data means.
Today Zhang San said that this customer number is a customer who has applied for a bank card, and tomorrow Li Si will say that he has borrowed a customer. In contrast, the field type and length of the two are the same, which opinion should be adopted?
Data standards are normative constraints that ensure the consistency and accuracy of internal and external use and exchange of data, and eliminate ambiguity through unified specifications.
1. Data governance implementation framework
The data governance system is an organization, process and tool established to standardize various management tasks and activities in business data specifications, data standards, data quality and data security.
Through a normalized data governance organization, establish a long-term mechanism for centralized data management, standardize the data management and control process, improve data quality, promote the consistency of data standards, and ensure the security of data sharing and use, so as to improve the operational efficiency and management level of enterprises.
2. Data governance organizational structure
In addition to the implementation architecture in terms of technology, the enterprise data governance system also needs the support of the organizational structure in terms of management.
Generally, in the early stage of data governance construction, the group will first set up a data governance management committee. From top to bottom, it is composed of decision-making level, management level, and executive level. The decision-making level makes decisions, the management formulates the plan, and the executive level implements it. Hierarchical management, unified coordination.
(1) Organizational structure
1) Decision-making level
Provide the decision-making function of data standard management, which is commonly understood as making a decision.
2) Management
- Review the relevant systems for the management of data standards
- Discuss and make decisions on cross-departmental difficult data standard management disputes
- Manage major data standard matters and submit them to the Information Technology Management Committee for deliberation
3) Execution layer
- Business department: responsible for the formulation, revision, and review of data standards for business lines, and the promotion and implementation of data standards
- Science and technology development: undertake the implementation of governance platform, data standards, data quality, etc., and follow data standards in system design and development
- Science and technology operation: responsible for the formulation of technical standards and technology promotion
(2) Management responsibilities
1) Project Manager
- Determine project objectives, scope, and plans
- Develop project milestones
- Manage cross-project collaboration
2) Expert review team
Review the project plan and determine the reasonableness of the plan
3）PMO
- Ensure that the project executes as planned
- Manage significant project risks
- Perform cross-project collaboration and communication
- Organize project key reviews
4) Data Governance Special Group
Implement the implementation and operation promotion of each project, and promote the implementation of data governance technology and project progress at the executive level.
(3) Executive responsibilities
Data architects, data governance experts, and business specialists form an "iron triangle" of data governance and work closely together to promote the implementation of data governance and data architecture.
1) Business Specialist
As the interface of data governance of the business department, the business specialist organizes the work of business personnel in the fields of standards, quality, and applications
- Define data rules
- Guarantee data quality
- Ask for data
2) Data Governance Specialist
As members of the data governance team, data governance experts are responsible for designing data architecture, operating data assets, and leading the organization of business and IT to achieve data governance goals.
- Build a logical model of your data
- Monitor data quality
- Operational data assets
3) Data Architect
As an expert in the IT development department, the data architect is responsible for the implementation of data standards and models, and assists in solving data quality problems.
- Data standards are implemented
- Logical model landing
- The physical model is on the ground
3. Data governance platform
After determining the technical implementation plan and organizational management structure, it is necessary to implement the data governance system.
In large enterprises, a complete data governance platform is generally developed, which includes all data governance functions and provides platform services to the outside world.
1) Core functions
As a product system of data governance, the data governance platform aims to ensure that the data of the data platform is safe, reliable, standardized, and valuable.
- Data asset management: Provides user-oriented scenario-based search and a panoramic data asset map to facilitate quick asset search and asset analysis
- Data standard management: Customize data standards in a unified manner, improve the management of fields, code values, and data dictionaries, and ensure the unified standards of business data and middle office data
- Data quality monitoring: Provides a data quality system before, during, and after the event, and supports functions such as data quality monitoring rule configuration and alarm management
- Data security: Provides data security masking, security classification, and monitoring
- Data Modeling Center: Unified modeling, providing business system modeling and model management
2) Metadata management
As the front-end display portal of the data governance platform, the metadata management system helps realize the ability to quickly retrieve data assets and improve the effectiveness and efficiency of data use.
By establishing a complete and consistent metadata management policy, it provides centralized, unified, and standardized metadata information access, query, and invocation functions.
3) Data quality
- Data quality monitoring: All users can configure data quality monitoring rules
- Rule blocking: Configure data quality monitoring blocking rules to block downstream jobs in real time when data quality discrepancies occur, and prevent the link spread of error results.
- Alarm: If there is a preset deviation in data quality, an early warning notification will be issued in time to fix it
4) Data standards
Support the customization of a unified data standard platform, including field standard management, code value standard management, dictionary management, and unified standards for business source data and middle platform data.
5) Data Security
Based on the group's data assets, it realizes hierarchical management of data security, automatically identifies security information, provides data access security behavior monitoring, and identifies access risks in a timely manner.
4. Data governance assessment
After the development and operation of the data governance platform, it is necessary to verify and evaluate the effectiveness of the overall data governance system.
"1) Whether the data can eliminate the phenomenon of "dirty, messy, and poor", 2) Whether the data assets are maximized, and 3) Whether the lineage of all data is complete and traceable ......"
1) Data assets
By building a data asset management system, it achieves full asset coverage and supports global search and accurate positioning of target assets.
- Implement global search and provide scenario-based search services for users
- Multiple search dimensions such as tags, data maps, table names, and field names are supported
- You can filter the results of data maps and source business data dictionaries
- For example, it supports PV/UV user search and asset display, and clarifies service goals
2) Data standards
The precipitation of new and old data standards has opened up data modeling tools, data standard databases and root standard databases, and implemented data standards and root standards.
- Achieve 100% pull-through of the data standard library
- Intelligently identify data standards and references
- The client synchronously updates the data standard and word root
3) Data Security
Maintain the principles of pre-event system construction, in-process technical control, and post-event monitoring and auditing, and establish a whole-process data security management and control system.
Based on the above data security management and control system, it supports data security grading and builds a flexible data security sharing process.
4) Data quality
Through the data quality radar chart, the data and task quality are scored on a regular basis, and the data quality effect is comprehensively investigated.
- Data Integrity: Check whether the data item information is comprehensive and complete
- Alarm response level: daily management, emergency response, impact reduction, and data damage and loss are avoided
- Monitor coverage: Ensure that data adheres to uniform data standards and specifications
- Job Stability: Monitors the job stability and whether there are any job exceptions
- Job timeliness: Check whether the data item information obtained by the task meets the expected requirements