Main contents of DOA

DOA mainly includes the following contents: the mechanism of data architecture, the composition of data architecture, the mechanism of data registration center, the mechanism of data authority center, the mechanism of data exception control center and the mechanism of data application unit.

(1) data architecture-oriented mechanism

Including: data-oriented, data-centered guiding ideology; The demand for architecture in the era of big data; Definition and classification of data; The carrier of data and its relationship with cloud computing; The meaning of data ecosystem and the role of DOA; The relationship between DOA and data and application; Data management and service mode of DOA; The relationship between DOA and application business logic and data logic; Basic principles of data security under DOA: and so on. Data-oriented and data-centered guiding ideology. Data is alive and has a life cycle, and the life process needs to be completely recorded. Data has attributes, such as security, identity, time and space. Data should be encrypted naturally, put on armor, and presented in encryption, with different encryption levels and depths. Data is independent of the system, data is the basis of application, independent of the specific hardware environment and software environment. The same data can support different applications. Data access and application are based on authorization, specific visitors, specific occasions (environment), specific time (period), data usage and AAA (authorization, authentication and accounting) mechanism that users are suitable for network security. Data is encrypted when stored and transmitted (data state) and decrypted for use after authorization (application state). The data system is ecological, changing and developing, sustainable, self-growing, self-managing and self-adapting. The virtual world is composed of data, which is the mapping of the real world. To establish a data ecosystem, we need to face the data and take the data as the core. Data ecosystem includes all kinds of ecological applications, and it is a "lush application forest growing on fertile data soil". The data ecosystem needs to build a logical data resource pool, support big data platforms and fragmented and growing applications, and support data sharing and system scalability. Based on data: everything can be measured, everything can be connected, everything can be operated and everything can be realized.

The demand for architecture in the era of big data. In the era of big data, we have moved from the information technology (IT) era to the DT era, data technology (DT). From focusing on technology (T) to focusing on information (I) and then focusing on data (D). Information varies from person to person, utilitarian and self-serving, emphasizing me and information technology for myself; Data is universal, can generate information, is public welfare, altruism, emphasizing that we, data technology is for everyone. Ma Yun pointed out that mankind has moved from the IT era to the DT era. The IT era is dominated by self-control and self-management, while the DT era is a technology that serves the public and stimulates productivity. Data technology, including information technology, has a wider scope and richer connotation; Information technology is more specific and targeted. Data system includes information system, which is a subset of data system. Cloud computing provides the possibility for the arrival of the era of data technology: cloud storage has almost unlimited mass data storage capacity. In the era of data technology, we need a new world view, that is, data world view and data security view. In the era of data technology, that is, the era of big data, a suitable software architecture is needed to support data security, support all business dataization, manage massive, heterogeneous, complex, changing and explosive growth big data, and provide support for mining valuable information. The existing system architecture is mostly the technical architecture left over and developed in the era of small data, and the existing security architecture is also built on the technical architecture of the era of small data. Technologies developed from information technology (small data era) and information security technology can no longer meet the requirements of big data era. Technologies such as Hadoop and MapReduce are only part of the solution to the static and massive problems of big data. It is necessary to re-examine the data and examine the technology, architecture and security system from the perspective of data.

Definition and classification of data. The definition of "data" given by Baidu Encyclopedia and Wikipedia respectively is: "Data is a numerical value, that is, the result we get through observation, experiment or calculation. There are many kinds of data, and the simplest one is numbers. Data can also be words, images, sounds, etc. Data can be used for scientific research, design, verification, etc. " Data, or data, refers to the symbolic records describing things, which can be defined as meaningful entities, and it involves the existing forms of things. It is a discrete and objective description of a group of events, and it is the original material that constitutes information and knowledge. Data can be divided into analog data and digital data. Data refers to' raw materials' processed by computers, such as graphics, sounds, characters, numbers, characters and symbols. In the era of big data, the data we study is generalized data: the content that the real world maps to the virtual world. In addition to the data that we understand that computers can directly process, they can also be all kinds of things that can be registered: such as devices, services, apps, people, things and so on. This requires studying the data definition or generalized data definition in the era of big data. Data can be classified from different angles: structured/unstructured data, relational database /NoSQL, dynamic data/static data, changing data/historical data, simple data/complex data, own data/* * shared data/public data, constantly changing and accumulating big data, etc.

The carrier of data and its relationship with cloud computing. Cloud computing can be divided into two types: Elastic Computing Cloud (EC2) and Amazon's Simple Storage Service (S3). Traditionally, the architecture is divided into three layers of IaaS, Infrastructure as a Service (IaaS), Platform as a Service (PaaS) SaaS and Software as a Service (SaaS). In essence, cloud computing is a cloud service, but it should also have an important DaaS layer, namely Data as a Service (DAAS). From the data point of view, cloud computing is more of a data storage service, namely IaaS and DaaS. It integrates infrastructure and data (I+D) through distributed and virtualization technologies, and provides flexible, measurable and personalized data and computing services for end users, which can be referred to as "cloud" for short. Everything is in the cloud, including all infrastructure, such as networks, servers, storage devices, and more importantly, all data, data in the online world and the physical world. Cloud is defined by data and can be divided into storage cloud, network cloud and physical cloud. Examples of storing cloud data: basic data, image data, historical data, industry data, etc. Examples of network cloud data: QQ, WeChat, Baidu, blog, SMS, Taobao, email, APP software, etc. Examples of physical cloud data: rainfall, temperature, video, PM2.5, traffic flow, equipment, people, etc. The meaning of data ecosystem and the role of DOA. Data ecosystem is a "lush application forest growing on fertile data soil", and DOA is a description of the sustainable development relationship between data and applications. The key to building a data ecosystem is to establish an effective mechanism, which is data-oriented architecture (DOA). DOA is an information system architecture, aiming at any data type, based on the concept of cloud service, independent of specific hardware platforms and software systems. With the concept of data as the core and data as the guide, the mechanism of building a complex information system is established, the data identification is used as the identification and positioning mark of data, the classification system and access rights of data are established, and data management and exchange are realized through data registration and registration centers, so that various data functional units can be established from simple to complex and from easy to difficult. The relationship between DOA and data and application. Humans have established a virtual world through computer software and hardware systems, including computer networks, and have recognized, constructed, transformed and adapted to the real world through the virtual world, thus producing a large number of data and various applications. DOA is a mechanism and platform established between data supported by cloud computing and various applications, which can manage and serve data and applications respectively, and form an ecosystem to cope with changing data and applications with the relative invariance of this mechanism and platform. This relationship and mechanism can also support from real-time data to real-time applications. The three-tier architecture consisting of data, DOA platform and application is shown in the right figure.

Data management and service mode of DOA. The data DOA faces is generalized data. To manage and serve generalized data, we must first solve the problem of unified identification and management of all kinds of data. Secondly, we should consider the value protection of data, the attributes of management data, and the authority and authorization of management data. Thirdly, in the case of distributed application and data redundancy, the consistency of data should be considered. Based on this, the data registration center (DRC), data permission center (DAC) and data exception control center (DEC) are proposed, which cooperate with each other to realize unified management of all kinds of data and provide data services for applications.

The relationship between DOA and application business logic and data logic. Traditional application information system construction logic is mostly business-oriented logic, that is, according to the requirements, according to the business process to analyze the requirements, and design and develop the system for this matter. According to business logic, it is necessary to design information processing flow and data structure according to the requirements of business flow. The advantage is that the information flow and business flow are relatively consistent, but the disadvantage is that once the business flow changes in the future, the information processing flow and data structure will also change, which will bring unpredictable difficulties to the system development and maintenance. DOA requires data orientation, that is, the business logic of the application should be transformed into data logic. In this way, it is required to classify business processes into small data-oriented processes according to the period of accessing the data resource pool, and finally integrate these data-oriented processes into business-oriented processes to complete the development of application information systems. The advantage of this is that once the data resource pool is built, it will be more convenient to build data-oriented business processes. Moreover, if the business process changes, it will not affect the whole data logic and data process, only need to increase the changed parts or adjust some data processes to adapt to the new changes. The disadvantage is that to convert business logic into data logic, you need to make an ideological change.

Basic principles of data security under DOA. Traditional information security, first of all, is to establish a closed and relatively safe environment, and to ensure that this closed environment is safe or credible through various means, but most of the data inside is "naked". Once an uninvited guest enters this environment through a loophole, "naked" data will face great danger. In the open environment of Internet and cloud computing, protecting data according to information security strategy will face great challenges.

DOA is data-oriented and data-centric. Data has attributes, including security attributes, identity attributes, time attributes and space attributes. Identify the owner, friends and enemies of the data. Considering the security problem from the point of view of data, we should ensure the integrity, confidentiality and availability of data. Data should be "naturally encrypted and authorized for use". Data has the function of self-protection, so it needs to be put on armor and presented in an encrypted way, with different levels and depths of encryption. The use of data should be authorized. There are two states of data: data state during storage and transmission and application state during authorized use. The "data state" is an encrypted state and the "application state" is a decrypted state. Once the "application" is completed or leaves the application environment, the data should be "changed" to the encrypted "data state" immediately. DOA provides encryption and decryption mechanism and authorized use mechanism, which makes data inaccessible and usable during storage and transmission, while authorized users are transparent when accessing or using data through applications, that is, they do not feel the encryption and decryption process of data. Therefore, the data security strategy under DOA is that "data status" data is suitable for both closed and open environments, and "application status" data is only suitable for "closed" environments. In this way, the data security problem is decomposed into several key issues, such as data encryption and authorization mechanism, data self-protection and automatic encryption and decryption mechanism, application environment security and so on.

(2) Composition of data-oriented architecture

Including the composition and completeness of DOA; Basic functions of data registration center (DRC); Basic functions of data rights center (DAC); Basic functions of data anomaly control center (DEC); Basic functions of dau; Preliminary study on the principle of data-oriented software engineering: etc.

Composition and integrity of DOA. DOA is a software architecture based on cloud computing environment, which does not involve direct control and access to specific hardware. As a mechanism to coordinate the relationship between data and applications and build a data ecosystem, DOA should have the function of comprehensive management and service for generalized data and various applications. The DOA components initially considered include: Data Registration Center (DRC) for registering and managing generalized and various data, Data Permission Center (DAC) for managing data authorization, authentication and billing (authorization and access process records), Data Exception Control Center (DEC) for managing data consistency, and Data Application Unit (DAUs) for managing various applications and providing services. These centers and application units constitute the basic framework of DOA. They are independent and interrelated, forming an organic whole. Independence is reflected in their different tasks and functions; The interrelationships are reflected in their interdependence. For example, DAC, DEC and dau all depend on DRC.

Basic functions of data registration center (DRC). The data registration center registers all kinds of data and generalized data to form a logical data resource pool, which is convenient for applications to access data. Its functions involve but are not limited to: definition of data registration information, data attribute information, data classification, metadata standard, metadata classification, registration methods of different types of data, data index, metadata index, data retrieval, generalized data pattern recognition, distributed deployment, on-demand adaptive mechanism of data registration content, automatic registration mechanism of data generation, historical data registration and management, etc.

Basic functions of data rights center (DAC). The data authority center manages data authority, and its functions involve but are not limited to: data security attribute definition, data legality authentication, data access authority definition, user authority authentication, application authorization, user authorization, data authorization and PKI, charging algorithm and mechanism, data transparent encryption and decryption mechanism, efficient data encryption and decryption algorithm, etc.

Basic functions of data anomaly control center. The data anomaly control center deals with data consistency when there is data redundancy in distributed environment, and its functions involve but are not limited to: data maintenance, adaptive management, anomaly detection and processing, patrol inspection, anomaly and conflict discovery, synchronization processing, redundancy processing, load balancing, etc. Basic functions of data application unit. Data application unit is a series of application unit modules in data resource Ikenoe. For application management and services, building blocks and application programming interfaces (APIs) similar to the component-based software development model (COA) are called to "data-driven applications" to quickly meet the various application functional requirements of users. Its functions should be based on the requirements of various specific applications, involving but not limited to: data function unit (DFU) that provides different functions according to different data types, data service unit (DSU) that provides services by push, data encryption unit (DEU), data authorization calling unit (DIU), data application combination unit (DCU), data visualization unit (DVU), data processing unit (DPU) and so on.

On the principle of data-oriented software engineering. DOA, a data-oriented architecture, provides a new method for software development. Different from the traditional business-oriented software engineering, the new data-oriented software engineering has new vitality. The following research will be carried out: the relationship between application software with life cycle and data ecosystem; Research on the development process from business logic to data logic: research on the construction and operation and maintenance mechanism of logical data resource base: research on the development mode of application software based on growing data ecosystem: research on the rapid construction mechanism of data-oriented application software based on DAUs: research on the data integration method of existing systems; Wait a minute.

(3) Data Registration Center (DRC) mechanism

Include data registration content and metadata standard definition; Data attribute information definition; Data classification and classification standards; Data registration method; Metadata indexing and retrieval methods; Generalized data pattern recognition; Distributed deployment mode of data registration center; On-demand adaptive mechanism of data registration content; Automatic registration mechanism of data generation; Registration and management of historical data; Wait a minute.

Definition of data registration content and metadata standard. Generalized data includes all kinds of data stored in the cloud, real-time changing data transmitted in the Internet, and data represented by physical objects and states existing in the physical world. If the concept of cloud is used to represent data, it means storage cloud (data), network cloud (data) and physical cloud (data). To register these data, it is necessary to define the registration content according to the characteristics of these data, and the most important thing is to point out the name and location of these data as the unique identification of unified management data. In addition, data description, data attributes, data permissions and other contents are required. All these contents are embodied in metadata, and it is necessary to formulate a unified metadata standard for data registration. Data attribute information definition. Data has attributes, and different data have different attributes. Data is valuable, and DRC needs to manage the essence of data. Such as data owner, data life cycle, data authority, data state, data nature, data legitimacy, data quality and so on.

Data classification and classification standards. Data can be divided into different categories and subcategories, and the standards, methods, categories and applications of classification need to be studied. In order to improve the efficiency of data retrieval, it is necessary to further classify metadata. Data registration method. According to different data types and data attributes, appropriate data registration methods should be adopted, which can be divided into manual registration, semi-automatic registration and full-automatic registration. At the same time of data registration, the data index is established. Applications generate data, and data generated by applications should be registered automatically.

Metadata indexing and retrieval methods. The data registration center provides data access services for applications, and the access efficiency depends on the indexing and retrieval methods. Because the volume of data registration center can be very large, its scale can reach TB level or even PB level according to different systems. Therefore, it is very necessary to establish an efficient metadata indexing and retrieval mechanism and study an efficient indexing and retrieval method. Generalized data pattern recognition. The content registered by the data registration center can be generalized data, such as entities in the physical world. In order to retrieve these generalized data quickly, we need to adopt new recognition technology. For example, pattern recognition technology based on fuzzy theory can be used to establish indicators and other methods. Distributed deployment mode of data registration center. Although the data in the data registration center is the registration information of data, its volume accounts for about one thousandth of the physical data. When the physical data reaches PB level, the data in the registration center will reach TB level. Therefore, the data registration center should also be deployed in the cloud distributed environment. In order to run the data registration center efficiently, it is necessary to study its distributed deployment mode.

(4) Data Rights Center (DAC) mechanism

DOA aims to design future data systems from the perspective of architecture, including data security. DAC protects data by managing data rights and provides a mechanism to authorize the use of data, which can also protect the interests of data owners. Therefore, the mechanism of DAC involves but is not limited to: the basic theory of data security in open environment; State mechanism of data; Inherent security attributes of data; Data access control authority and management mechanism; Identification of data legitimacy; The function and operation mechanism of data authority center; User authentication mechanism and CA technology; Data authorization mechanism and its relationship with PKI and public key infrastructure (PKI); Data usage record and its traceability mechanism; Accounting mechanism; Multi-level authorization and authentication mechanism; Authorization mechanism for single data and batch data or large data volume; Key systems; Encryption and decryption strategies and algorithms for data transparency; Compromise between encryption and decryption efficiency and security and authorization process; Adaptability of traditional data transmission encryption technology; Application of environmental safety; Identify illegal use of data and digital watermarking technology; Issues related to the rights and intellectual property rights of data owners; Wait a minute.

Basic theory of data security in open environment. In an open environment, to make the data itself safe and safe to use, we must first encrypt the data, which should have the characteristics of "natural encryption and authorized use". Let's assume that data is unencrypted when it is used, so data should remain encrypted when it is not used. Therefore, the set data has two states: a data state in an encrypted state when it is stored and transmitted, and an application state in a decrypted state when it is authorized for use. As a mechanism, DOA should ensure that data can be associated with authorization and encryption and decryption technologies in these two states. At present, the theory and method system of data security, AAA technology, CA technology, PKI technology, key system, encryption and decryption technology, as well as network security technology, system security technology, application environment security technology and so on. All of them are applicable, but they should be reorganized from a data-oriented and data-centered perspective, and adaptive research and improvement should be carried out from the aspects of data security concepts, theories, methods and application mechanisms for protecting data.

State mechanism of data. You can view data from an object-oriented perspective. In addition to its own value, data also has internal attributes and external states. Externally, data should have two states: data state and application state. It is necessary to study the definition, setting and acquisition of data state, the transformation of data state, the function of data state, the requirements of data state on environment, the relationship between data state and data encryption and decryption, and the mechanism of action.

Inherent security attributes of data. The internal attributes of data include key data security attributes. To study the definition, content, access rights, data security description, data status, data owner (data owner), friend (authorized person), stranger (unauthorized person), enemy (unauthorized person), data reading and writing rights, data attachment history, data digital watermarking and anti-counterfeiting identification, data authorization records, etc.

Data access control authority and management mechanism. Data access control depends on data security attributes and is closely related to data encryption and decryption. In the past, data access control authority was controlled by software, and access software controlled data access, and the data itself may or may not be encrypted. When another software accesses data, it may bypass access control, such as unauthorized access to data, resulting in illegal access to data and disclosure of important information. This research is based on the concept that the data itself is encrypted and authorized for use. Data access is based on the security attributes of data and the identity of visitors, and then the data authorization is determined by application authorization and user authorization. According to the authorization mode and application environment, provide decryption key or decryption algorithm to realize the safe use of data. It involves the use of data, the right to read and write data, the right to modify data, the right to add data, the acquisition of data and the determination of data owner, as well as the automatic encryption and automatic registration of data according to the identity information of data owner. According to different application types, different application scenarios and different user expressions, this data access control method and authority management mechanism need further study.

Function and operation mechanism of data center. The data authority center is responsible for data security protection, authorization management of data use and application security management. Therefore, the data authority center should manage the security attributes of data, identify the legitimacy of data, set the access authority of data, authenticate users and applications, authorize users and applications of data, record and account the authorization process, encrypt and decrypt data, and so on. The data authority center should cooperate with the data registration center, and the data involving data attributes and rights need to be registered and registered in the data registration center. The data authorization center will monitor, authorize, revoke authority, authenticate, keep accounts, encrypt and decrypt data according to the registration information, and register new data security attributes. Internally, the process of using data is the process of data authorization and expanding the scope of authorization, and accounting is the record of these authorizations, which can lay the foundation for subsequent commercial applications. Any DOA platform in the future should not only provide data management and services, but also have the basic ability of data business operation.

User authentication mechanism and certificate authority (CA) technology. Data application authorization is based on user authentication. User authentication is related to the attributes of users, and the authentication process is the process of user registration, management and maintenance. Registered user information is an important data registration content of the Data Registration Center (DRC) and also an important data needed by the Data Rights Center (DAC). The user authentication technology can adopt the traditional CA technology, which requires a third-party authority center or a local center to issue a user certificate (private key) to the user. At the same time, the relationship between data and users is established through data security attributes.

Data authorization mechanism and its relationship with public key infrastructure. To authorize users to use data, it is necessary to convert the data encrypted by the public key of the data owner (the owner of the data) into the data encrypted by the public key of the authorized user (the friend of the data), and then provide it to the authorized user for download and use. When encountering a large amount of data, in order to improve the efficiency of encryption and decryption, it should be the key that encrypts the data symmetrically, not the data itself. Data Rights Center (DAC) should provide PKI-based encryption and decryption authorization mechanism and method.

(5) data anomaly control center mechanism

DEC manages the consistency of data resource pool (data registered by data registration center), maintains data resources and ensures the uniqueness and consistency of data. Therefore, the mechanism of DEC involves but is not limited to: data consistency maintenance mechanism, data dynamic change adaptive management mechanism, data consistency anomaly detection and processing mechanism, data patrol algorithm, data anomaly and conflict discovery algorithm, data synchronization processing algorithm, data redundancy processing algorithm, hot data automatic replication technology, cold data automatic deletion technology, system load balancing and so on.

(6) DAUs mechanism

DAUs has established a series of application unit modules on the basis of data registration center DRC and data authority center DAC. Through building blocks similar to component-based software development model (COA) and calling application program interface (API), Daus can "data-driven application", quickly meet various application function requirements of users, and manage and serve various applications. Therefore, the mechanism of DAUs involves but is not limited to: data application unit structure specification, program call parameter specification, data access specification, application registration management specification, application extension mechanism, authorized data access mechanism, unauthorized data access identification, data function unit, data service unit, data encryption and decryption unit, data authorized call unit, data application combination unit, data visualization unit, data processing unit, etc.