Increased load on the hosting due to requests from foreign IPs

Started by Ronny, Oct 19, 2022, 12:47 AM

Previous topic - Next topic

RonnyTopic starter

Good day!
On October 15-16th the load on hosting has risen sharply (5-6 times). The hoster's message is as follows:
QuoteThere are a lot of requests to the site from many foreign IP-addresses. To block such requests and reduce the load you can activate access to the site only from our country in the control panel.
The easiest option - restriction of access only from the country (it can be temporarily activated), because the requests use the legitimate UserAgent, and IP-addresses and URLs in queries are different. That is, there is no common parameter by which to selectively block requests.
You can connect the domain to Cloudflare: https://www.cloudflare.com/ , perhaps it will help filter out such requests.
While I activated the tool access only from my country, but I do not want to leave it this way, after all 13% of foreign traffic. I have no experience and skills in solving such problems, I have never faced with this. Who can give some advice in this situation? Preferably, as for "dummies", so to speak.
  •  

Newport

Set the anti-bot. At -DM- somewhere it was.
And maybe it's such a hosting, which does not keep attendance?
I once had a hoster wrote that there was a serious attack on the site, by a serious attack meant the visits of googlebots, bing, and robot sapa )).
  •  

-DM-

Quote from: Newport on Oct 19, 2022, 02:34 PMSet the anti-bot. At -DM- somewhere it was.
and cloudflare by these rules wmsn.biz/m.php?p=143697

Cloud Flare FirewallRule setup from the 3 rules, which will work for many, but not all.
I try to set up all my sites this way now. For me, it's convenient.

Start by enabling forced redirect from http to https, you can find it in: SSL/TLS | Edge_Certificates | Always_Use_HTTPS.
Then in Firewall - Firewall_Rules create rules (they should be in this order):

1) Allow access to white bots. Who CloudFlare considers white listed here.
If the bots get to http - will have a 301 redirect https.
The rule looks like this: (cf.client.bot)
You cannot view this attachment.
2) Those who got to http - make 5-second JS check (if there are attackers), let them suffer, after that they will get redirect to https.
If you do not make this rule, then http will not be available at all because of rule number 3.
The rule looks like this: (not ssl)
You cannot view this attachment.
3) Deny access to everyone who visits not via HTTP2 protocol, dhdos-bots, shitbots, and other stuff that goes mainly via HTTP/1.0 and HTTP/1.1.
The rule looks like this:
You cannot view this attachment.
And a little bit about the sad part. HTTP2 is not supported by a couple percent of browsers, mostly these
old abandoned mobile browsers, but they are as good as milk, these browsers are not convenient to use and
the hell to do with them on the site, no goods to order, or advertising to click, or comments to shit. Although,
those who are still optimizing sites for ie7, - for these webmasters is probably a loss.

These settings do not obviate the need to use antibots, as Ahrefs, Archive, baidubots and other unnecessary bots
for CloudFlare are considered white and they pass. Also, many hotbots use HTTP2, which abuzayut, etc.
are negative activity against sites, from them will save only cloud antibot.
  •  

Newport

By the way, I also in tech support recently recommended to block IP....Google, Bing, Yandex, and ArchiveOrg  ;D 
Well, morons, no words, only emotions, bordering on hysteria...
  •  

Kurz

Quote from: Newport on Oct 19, 2022, 02:34 PMby a serious attack meant the visits of googlebots, bing, and robot sapa )).
Now most of the "shared" started tightening the screws, the crisis, the equipment is getting more expensive, so they include all sorts of "protections" that harm the site, but the server loads less to the hoster.
  •  

RonnyTopic starter

Quote from: Newport on Oct 19, 2022, 02:34 PMI once had a hoster wrote that there was a serious attack on the site, by a serious attack meant the visits of googlebots, bing, and robot sapa )).
Hoster, I wrote myself with a request to advise the causes of such a sudden jump in load, which I noticed in the control panel.
Observed it for 2 days, before that everything was normal. Attendance is at the same level.
There are several days in a year when the traffic is increasing. But even on those days the load is much less than the current values.

Quote from: -DM- on Oct 20, 2022, 09:27 PMand cloudflare by these rules wmsn.biz/m.php?p=143697
That is, only your anti-bot will not be enough?
  •  

-DM-

Quote from: Ronny on Oct 21, 2022, 06:12 AMThat is, only your anti-bot will not be enough?
Antibot works on your server, culling primitive bots by claudflare outside your server is a significant resource saver.
I do not know how to live without claudflare, it's awesome, convenient dns, saving traffic and server resources, ssl, and much more.
  •  

RonnyTopic starter

Quote from: -DM- on Oct 21, 2022, 06:39 AMAntibot works on your server, culling primitive bots by claudflare outside your server is a significant resource saver.
I do not know how to live without claudflare, it's awesome, convenient dns, saving traffic and server resources, ssl, and much more.
Maybe, I do not argue. I just never got into the subject of bots because I didn't have that problem. It all seems complicated and convoluted to me.
Any step by step instructions on how to properly connect and configure it all?
  •  

Kurz

Yeah, had the same problem this summer. A very uncomplicated and inexpensive way, is to move all sites to work through cloudflare. Turn on high security mode.
It's likely that requests from bots can go not only to the sites, but also to the server IP. You need to close access for requests that are not coming through CF.
Cloudflare is also useful in that it spoofs the true IP address of the host, which will help avoid problems in the future.
  •  

Ali_Pro

You set up a CF. There are 5 free rules will be enough for you.
How you hook it up, you'll see that most bots come from amazon.com, digitalOcean, hetzner, ovh.
They are immediately under the captcha.
And that's where your whole epic may be over.
Ali.
  •