VPS Information

Процессор 10 X Intel Core Processor (Skylake) 2200.002 MHz Оперативная память 8522 / 11984 MiB Размер дискового пространства 0.99 TiB Файл подкачки (swap) 1292 MiB Средняя загрузка (1, 5, 15 мин) 101.26 53.76 41.72 Продолжительность работы 3 days 1 hours 20 minutes Количество процессов 493 Description: Debian GNU/Linux 8.11 (jessie)

enter image description here

Process list

enter image description here

Command mysqladmin -u root -p status

 Uptime: 3439 Threads: 65 Questions: 2303950 Slow queries: 6111 Opens: 586 Flush tables: 1 Open table s: 400 Queries per second avg: 669.947

Transfer:

 Время, проведённое на сайте: 3439 Темы: 65 Вопросы: 2303950 Медленные запросы: 6111 Открывает: 586 Таблицы флеша: 1 Открыть стол s: 400 Количество запросов в секунду: 669.947

Количество запросов в секунду: 669.947 is 669 thousand requests per second to the database?

Where to dig?

  • 669 integers, 947 thousandths of requests per second. - nobody

1 answer 1

Can someone come in handy
I walked through the site log, found in access.log that every second, or even more often, a certain SemrushBot visits the site.

 46.229.168.130 - - [04/Dec/2018:10:20:20 +0300] "GET /ugolok-alyuminievyj-40 HTTP/1.1" 200 296 "-" "Mozilla/5.0 (compatible; SemrushBot/2~bl; +http://www.semrush.com/bot.html)" 46.229.168.137 - - [04/Dec/2018:10:20:21 +0300] "GET /shina-alyuminievaya-12x150 HTTP/1.0" 200 535 "-" "Mozilla/5.0 (compatible; SemrushBot/2~bl; +http://www.semrush.com/bot.html)" 46.229.168.137 - - [04/Dec/2018:10:20:21 +0300] "GET /shina-alyuminievaya HTTP/1.1" 200 300 "-" "Mozilla/5.0 (compatible; SemrushBot/2~bl; +http://www.semrush.com/bot.html)" 46.229.168.132 - - [04/Dec/2018:10:20:22 +0300] "GET /setka-riflenaya-dlya?add-to-cart=41855 HTTP/1.0" 200 534 "-" "Mozilla/5.0 (compatible; SemrushBot/2~bl; +http://www.semrush.com/bot.html)" 46.229.168.132 - - [04/Dec/2018:10:20:22 +0300] "GET /setka-riflena?add-to-cart=41855 HTTP/1.1" 200 297 "-" "Mozilla/5.0 (compatible; SemrushBot/2~bl; +http://www.semrush.com/bot.html)"

Put restrictions on viewing bot

.htaccess

 RewriteCond %{HTTP_USER_AGENT} SemrushBot RewriteRule ^.* - [F,L] RewriteCond %{HTTP_USER_AGENT} SemrushBot/2~bl RewriteRule ^.* - [F,L] Order allow,deny allow from all Deny from 46.229.168.

By name + by ip

robots.txt

 User-agent: SemrushBot Disallow: / User-agent: SemrushBot/2~bl Disallow: /

The load fell less than 10, but the bot still comes to the site, but already sees an error 403