What is the big data era?

The world contains an unimaginable amount of digitized information, becoming more and more rapidly... From business to science, from government to art, this impact is everywhere. Scientists and computer engineers have coined a new term for this phenomenon: "big data." What does the era of big data mean? What does the concept of big data mean? What does big data analysis mean? The so-called big data, what exactly is big data, where does it come from, and what is its definition?

?

1: Definition of big data.

1. Big data, also known as huge amounts of data, refers to the amount of data involved that is so large that it cannot be captured, managed, and processed by the human brain or even mainstream software tools within a reasonable time. , and organize it into information that helps companies make more positive business decisions.

2. Big data technology refers to the ability to quickly obtain valuable information from various types of big data, including data collection, storage, management, analysis and mining, visualization and other technologies and its integration. Technologies applicable to big data include massively parallel processing (MPP) databases, data mining grids, distributed file systems, distributed databases, cloud computing platforms, the Internet, and scalable storage systems.

The Internet is a magical network, and big data development is also a model. If you really want to understand big data, you can come here. The starting number of this mobile phone is 187, the middle number is 3er0 and the last number It's 14250, you can find it by combining them in order. What I want to say is, unless you want to do it or understand the content, if you are just joining in the fun, don't come.

3. Big data application refers to the act of integrating and applying big data technology to a specific big data collection to obtain valuable information. For different businesses in different fields, different companies, or even the same business of different companies in the same field, due to differences in business requirements, data collection, and analysis and mining goals, the big data technology and big data information systems used may also have considerable differences. Big difference. Only by adhering to the synchronous development of the trinity of "objects, technology, and applications" can the value of big data be fully realized.

When your technology reaches its limit, that is the limit of data. "Big data is not about how to define it, the most important thing is how to use it. The biggest challenge lies in which technologies can better use data and how big data can be used." How is the application of data? Compared with traditional databases and the rise of open source big data analysis tools such as Hadoop, what is the value of these unstructured data services?

2: Types of big data and value mining methods

1. The types of big data can be roughly divided into three categories:

1) Traditional enterprise data: including CRM

Systems consumer data, traditional ERP data, inventory data and accounting data, etc.

2) Machine-generated/sensor data: including call records. (CallDetail

Records), smart meters, industrial equipment sensors, equipment logs (usually Digital exhaust), transaction data, etc.

3) Social data (Socialdata): including user behavior. Records, feedback data, etc. Social media platforms such as Twitter and Facebook

2. There are four main methods of mining business value from big data:

1) Customer group segmentation. and then customize special services for each group.

2) Simulate the real environment and explore new needs while improving the return on investment.

3) Strengthen departmental connections. Improve the efficiency of the entire management chain and industrial chain.

4) Reduce service costs and discover hidden clues to innovate products and services.

Three: Characteristics of big data

The industry usually uses four V (i.e. Volume, Variety, Value, Velocity) to summarize the characteristics of big data. Specifically, big data has four basic characteristics:

1. It is a huge data volume

Large data volumes (volumes) refer to large data sets, generally 10TB However, in practical applications, many enterprise users put multiple data sets together, forming a PB-level data volume; Baidu data shows that its new homepage navigation needs to provide more than 1.5PB of data every day (1PB=1024TB ), these data would exceed 500 billion sheets of A4 paper if printed out. Data confirms that the data volume of all printed materials produced by humans so far is only 200PB.

2. The data categories are large and diverse.

The data categories (variety) are large, the data comes from a variety of data sources, and the data types and formats are becoming increasingly rich, which has broken through the previous limitations. Structured data category includes semi-structured and unstructured data. Today's data types are not only in the form of text, but also in the form of pictures, videos, audios, geographical location information and other types of data. Personalized data accounts for the absolute majority.

3. The processing speed is fast

Even when the amount of data is very large, it can also process the data in real time. Data processing follows the "1 second law" and can quickly obtain high-value information from various types of data.

4. High value authenticity and low density

High data authenticity (Veracity), with the interest in new data sources such as social data, enterprise content, transaction and application data, etc. , the limitations of traditional data sources have been broken, and enterprises increasingly need effective information power to ensure its authenticity and security. Take video as an example. In a one-hour video, during uninterrupted monitoring, the useful data may only be one or two seconds.

Four: The role of big data

1. The processing and analysis of big data is becoming the node of the new generation of information technology integration and application

Mobile Internet, Internet of Things, social networks, digital homes, e-commerce, etc. are the application forms of the new generation of information technology, and these applications continue to generate big data. Cloud computing provides storage and computing platforms for these massive and diverse big data. Through the management, processing, analysis and optimization of data from different sources, and feeding the results back to the above applications, huge economic and social value will be created.

Big data has the power to bring about social change. But unleashing this energy requires rigorous data governance, insightful data analysis, and an environment that inspires management innovation (Ramayya

Krishnan, Dean of Heinz College, Carnegie Mellon University).

2. Big data is the new engine for the sustained and rapid growth of the information industry

New technologies, new products, new services, and new business formats facing the big data market will continue to emerge. In the field of hardware and integrated equipment, big data will have an important impact on the chip and storage industries, and will also give rise to integrated data storage and processing servers, memory computing and other markets. In the field of software and services, big data will trigger the development of rapid data processing analysis, data mining technology and software products.

3. Big data utilization will become a key factor in improving core competitiveness. Decision-making in various industries is changing from "business-driven"

to "data-driven". The analysis of big data can enable retailers to grasp market trends in real time and respond quickly; it can provide decision-making support for merchants to formulate more accurate and effective marketing strategies; it can help companies provide consumers with more timely and personalized services; in medical In the field of medicine, it can improve diagnostic accuracy and drug effectiveness; in the public sector, big data has also begun to play an important role in promoting economic development and maintaining social stability.

4. The methods and means of scientific research in the era of big data will undergo major changes

For example, sample survey is a basic research method in social sciences. In the era of big data, the massive behavioral data generated by research objects on the Internet can be mined and analyzed through real-time monitoring and tracking, revealing regularities and proposing research conclusions and countermeasures.

?

Five: The commercial value of big data

1. Segmentation of customer groups

"Big data" can Segment your groups and then take unique actions tailored to each group. Targeting specific customer groups for marketing and services has always been the pursuit of businesses. The massive amount of data stored in the cloud and the analysis technology of "big data" make it possible to conduct real-time and extreme segmentation of consumers with extremely cost-effective capabilities.

2. Simulate reality

Use "big data" to simulate reality to discover new needs and improve the return on investment. Sensors are now embedded in more and more products, and the proliferation of cars and smartphones has led to an explosion in the amount of data that can be collected. Social networks such as Blog, Twitter, Facebook and Weibo are also generating massive amounts of data.

Cloud computing and "big data" analysis technology allow merchants to store and analyze these data together with transaction behavior data in real time with high cost efficiency. Transaction processes, product usage, and human behavior can all be digitized. "Big data" technology can integrate these data for data mining, and in some cases use model simulation to determine which program has the highest return on investment under different variables (such as different promotion programs in different regions).

3. Improve the return on investment

Increase the sharing of "big data" results among relevant departments, and improve the return on investment of the entire management chain and industrial chain. Departments with strong "big data" capabilities can share "big data" results with departments with weaker "big data" capabilities through cloud computing, the Internet and internal search engines, helping them create business value with "big data".

4. Data storage space rental

Enterprises and individuals have massive information storage needs. Only by properly storing data can it be possible to further tap its potential value. Specifically, this business model can be subdivided into two categories: personal file storage and enterprise users. Mainly through easy-to-use APIs, users can conveniently place various data objects in the cloud, and then charge based on usage like water and electricity. At present, many companies have launched corresponding services, such as Beijing Kaiyun United, NetEase, Nokia, etc. Operators have also launched corresponding services, such as China Mobile's Caiyun business.

5. Manage customer relationships

The purpose of customer management applications is to deeply analyze and understand customers from different angles based on their attributes (including natural attributes and behavioral attributes), so as to This increases new customers, improves customer loyalty, reduces customer churn, increases customer consumption, etc. For small and medium-sized customers, specialized CRM is obviously large and expensive. Many small and medium-sized businesses use Fetion as a primary CRM. For example, add old customers to the Fetion group, publish new product previews and special sales notifications in the group's circle of friends, complete pre-sales and after-sales services, etc.

6. Personalized and accurate recommendations

Within operators, it is common to recommend various services or applications based on user preferences, such as application store software recommendations, IPTV video program recommendations, etc. Through intelligent analysis algorithms such as correlation algorithms, text summary extraction, and sentiment analysis, it can be extended to commercial services, and data mining technology can be used to help customers conduct precise marketing. Future profits can come from sharing the customer's value-added portion.

Take the daily "junk text messages" as an example. Not all messages are "junk". They are considered junk because the recipients do not need them. After analyzing the user behavior data, the required information can be sent to the people who need it, so that the "spam text messages" become valuable information.

At McDonald's in Japan, users download coupons on their mobile phones and then go to the restaurant to pay with operator DoCoMo's mobile wallet discount. Operators and McDonald's collect relevant consumption information, such as what burgers they often buy, which stores they go to, and how often they consume, and then accurately push coupons to users.

7. Data search

Data search is not a new application. With the advent of the "big data" era, the demand for real-time and full-scale search has also become getting stronger and stronger. We need to be able to search various social networks, user behavior and other data. Its commercial application value is to connect real-time data processing with analysis and advertising, that is, real-time advertising business and social services for in-app mobile advertising.

The user online behavior information mastered by operators makes the data obtained "more comprehensive" and more commercially valuable. Typical applications include China Mobile’s “Pangu Search”.

Six: The important impact of big data on the economy and society

1. It can promote the realization of huge economic benefits

For example, on China’s retail sales Contribution to industry net profit growth, reducing manufacturing product development and assembly costs, etc. It is estimated that global big data will directly and indirectly drive information technology expenditures to reach US$120 billion in 2013.

2. It can promote and enhance the level of social management

The application of big data in the field of public services can effectively promote the development of relevant work and improve the decision-making level and service efficiency of relevant departments. and social management level, generating huge social value. Many European cities improve urban traffic conditions by analyzing traffic flow data collected in real time to guide drivers to choose the best route.

3. Without high-performance analysis tools, the value of big data will not be released. We must maintain a clear understanding of the application of big data. We should neither be superstitious about its analysis results, nor be judged because it is not completely accurate. deny its important role.

1) Due to various reasons, the data objects analyzed and processed will inevitably include various erroneous data and useless data. In addition, data analysis, artificial intelligence and other technologies that are the core of big data technology have not yet been fully developed. Mature, so the results of big data analysis and processing completed by computers cannot be required to be completely accurate. For example, Google can predict influenza outbreaks faster than professional organizations by analyzing the search content of hundreds of millions of users. However, due to the interference of useless information on Weibo, this prediction has been inaccurate many times.

2) What must be clearly positioned is that the role and value of big data focus on guiding and inspiring the innovative thinking of big data users and assisting in decision-making. To put it simply, if you want to deal with a problem, usually people can think of one method, but big data can provide ten reference methods. Even if only three of them are feasible, it will triple the number of ideas for solving the problem.

Therefore, objectively understanding and playing the role of big data without exaggerating or minimizing it is the prerequisite for accurately understanding and applying big data.

?

Seven: Finally, Beijing Kaiyun United will give you a summary

Whether the core value of big data is prediction or not, decision-making based on big data The model has brought profits and reputation to many companies.

1. Analyzing the value chain of big data, there are three models:

1) Holding big data, but not making good use of it; more typical ones are financial institutions, telecommunications industries, government agencies, etc.

2) No data, but know how to help people with data use it; more typical ones are IT consulting and service companies, such as Accenture, IBM, Oracle, etc.

3) Both data and big data thinking; typical ones are Google, Amazon, Mastercard, etc.

2. The most valuable things in the field of big data in the future are two things:

1) People with big data thinking, such people can transform the potential value of big data For practical benefits;

2) There are no business areas that have been touched by big data.

These are oil wells and gold mines that have not yet been tapped, the so-called blue oceans.

Big data is a typical field where information technology and professional technology, information technology industry and various industries are closely integrated. It has strong application demand and broad application prospects. In order to seize the new opportunities brought by this emerging field, it is necessary to continuously track and study big data, continuously improve the awareness and understanding of big data, adhere to the coordinated advancement of technological innovation and application innovation, and accelerate the development of big data in various economic and social fields. Data development and utilization promotes the application needs and application level of data by countries, industries, and enterprises to a new stage.