A few days, maybe almost a week, I fed the guys (Yandex and Google) through their ping services of 10 million records.
Today we find that customers complain about the speed of the software, which refers to our server.

I could not find the problem on my own, the senior admin of the 80th level helped.
It was found that the temporary directory (100MB) files from the sessions was jammed.

Further, in the logs I found an appeal to those pages that fed Yandex and Google. But the message went (checked 5 addresses) from the Chinese IP.

Actually questions / s:

From where did the Chinese botnet find out about the pages that are not yet in the search?
Maybe this is a Yandex or Google botnet, because you need to process 10 million, and this is just mine?
Maybe someone of them broke a server that collects pings?

At the moment, it seems logical, forbidden to create a session (in the script), if there is a call to that section without a referral.
These links are created to feed the search engines, so that people go over the search, which means there must be a referrer.

Who does what in such situations?

  • one
    Yasha is Yandex? Or yahoo. - Vladimir Gamalyan
  • Corrected. (Yandex) - borodatych
  • no need to think that someone leaked your server to the Chinese. The Chinese stupidly scanned with whole subnets. I so raised my owncloud at home. And in a day, the Chinese are breaking up, trying to make a federation (that is, to connect with mine). - KoVadim
  • I readily believe, but it is difficult to guess such a link /items/ACI+-+AVESA/IT4918901370 . I looked at about 10 pages that were addressed, all working, there are no such /items/НЕПОНЯТНОЧТО/НЕПОНЯТНОЧТО - THING - borodatych
  • Are there no links to these addresses? - KoVadim

1 answer 1

Take logs and see who loads you. Your server - your logs.

There are many search engines that put a bolt on the site’s recommended by the webmaster’s delay between scans - and they are usually immediately ignored, because the traffic from them is miserable, and the load of hoo. Typical practice - immediately to ignore (Baidu, MSN - each has its own preference).

The same practice is to cut countries by IP, of which there will be no buying traffic. Africa, Asia, China - also to taste. GEO IP base doesn’t need to be constantly updated.

If after this you still want to cut something, you can simply reduce the length of the session. If you set up a session a day or a week, then the server will store all unexpired sessions. Reduce, this is also the normal direction of optimization.

But first - in the log. In order not to build a hypothesis "these are the bots of the bots who slip the links and DDoSyat" and "this is probably a Chinese search engine" and so on and so on And then you can fantasize endlessly and fruitlessly.

PS How do they know about the pages - yes, it may be legally turned to sitemap.xml - and you are wondering, who hasn't hacked you, hasn't anyone hacked Google ...

  • I wrote that the Chinese. Just not clear from where they found the links. Maybe in Google or Yandex where is the backdoor, from where does the drain go? - borodatych
  • No, there is no card as such. - borodatych