Crawl the website with a crawler, will the other party notice?

If the other party does not deliberately test, it will not be detected.

Simply put, reptiles are detection machines. Its basic operation is to simulate human behavior, go to various websites, click buttons, check data, or recite the information you see. Like a bug crawling around the building tirelessly. Baidu, which we use every day, actually uses this crawler technology.

I post countless reptiles to various websites every day, get their information back, then put on light makeup and wait in line for you to find it. Another example is the ticketing software, which is equivalent to throwing out countless avatars, and each avatar is helping you constantly refresh the train tickets on the 12306 website. Once you find the ticket.

I'll shoot it right away, and then I'll shout to you: local tyrants come and pay. What do you know about reptiles? Crawlers can also be divided into good and evil crawlers: like the crawlers of search engines, the crawled websites get free exposure opportunities, which bring traffic to the websites, and the crawled parties are also willing and happy to find data.

Therefore, this crawler is well-intentioned and crawls web data (such as URL) in strict accordance with the Robots protocol.