New Function: Alibaba Cloud Anti-Crawler Management Tool!

The situation of reptiles

Web security situation is not optimistic. According to the robot report of global dots 2065 438+08, reptiles account for about 42% of web page traffic.

Why climb backwards?

Prevent excessive consumption of resources

A large number of robots visit websites. Imagine that 42% of the traffic on your website is not visited by real people. A considerable part of them will also occupy a lot of network bandwidth, server computing and storage resources in the background.

scalper

Airline seat occupation: scalpers use malicious crawlers to traverse low-priced air tickets, and at the same time launch machines in batches to request seat occupation, which leads to the continuous occupation and waste of flight seat resources, and finally leads to high flight vacancy rate, which causes business losses to airlines and damages the interests of normal users.

Anti-bonus hunter Party

In e-commerce activities, scalpers launch machine requests in batches to simulate normal transactions, such as time-limited spikes and preferential activities for high-value goods, and then resell goods and resources to earn the difference, resulting in the marketing resources of e-commerce enterprises not reaching normal users and being profiteered by scalpers.

Anti-hacker

Core interface is brushed: login, registration, SMS and other business links are key nodes in the business, and related interfaces are often used by hackers to prepare for subsequent fraud.

Private letter rookie 007 can get dozens of sets of PDF!

Why do you need log analysis?

Find the robot that is hidden deeper.

Reptiles and anti-reptiles are an offensive and defensive process. According to the aforementioned report, advanced robots account for 74% (the rest are relatively simple robots), while according to the report of fileye M-Trends 20 18, it takes as much as1Kloc-0/day for the attacks of general enterprises to be discovered, among which the problem in the Asia-Pacific region is more serious, and general network attacks are found. With the log, we can better find out the hidden bad robots.

Understand robots and treat them differently.

Crawlers are also divided into good and bad, and search engines can query them, thus achieving SEO effect and bringing more valuable visits. Logs can help administrators to better distinguish which robots are good, and make more appropriate anti-climbing configuration according to them.

Keep reporting evidence

Robots that find illegal attacks can keep the attacker's information and path as important evidence for alarm.

Improve operation and maintenance efficiency

According to the log, we can find the abnormality and give an alarm and take action quickly.

More additional functions

Depending on other functions of the log service, it can play a greater role in the log.

Anti-Climbing Management in Alibaba Cloud —— Overview of Real-time Log Analysis

Alibaba Cloud anti-climbing management

Yundun anti-Bot service is a network application security protection product, which specializes in detecting advanced crawlers and reducing the business impact of crawlers and automation tools on websites. The product provides a set of malicious Bot protection solutions from Web, App to API interface to avoid the security shortcomings caused by poor protection in a certain link.

Alibaba Cloud log service

Alibaba Cloud's log service is a one-stop service for log data, which can quickly complete the collection, consumption, delivery, query and analysis of massive log data without development, and improve the efficiency of operation and maintenance. Log service mainly includes real-time collection and consumption, data delivery, query and real-time analysis, and is suitable for all kinds of development, operation and maintenance, operation and security scenarios from real-time monitoring to data warehouse:

At present, WAF in Alibaba Cloud is connected with log service to develop Web access and attack logs. Provide near-real-time automatic collection and storage of website-specific logs, and provide query analysis, alarm reporting, downstream computing docking and delivery capabilities based on log services.

Release area

Applicable customers

Functional advantage

The anti-crawling log real-time query and analysis service has the following functional advantages:

Open premise

Restriction statement

The log library stored in the anti-climbing management belongs to the exclusive log library, with the following restrictions:

usage scenario

1. Track the crawling and blocking logs of robots and track security threats;

Check the list of crawling robots of Top 100:

2. Real-time normal trusted Web request activity, insight into status and trends:

SQL for viewing PV/UV access trends:

3. Quickly understand the efficiency of safe operation and give immediate feedback:

SQL for viewing trends in valid requests and interception rates:

4. Output the security network logs to the self-built data and calculation center.

Further reference

We will release the best time for WAF security log analysis one after another. Please refer to the relevant user manual for more information: