During my time working as a technical support engineer in a hosting company, I encountered various server load problems such as DDoS attacks that would block websites and cause the server to come to a halt.



 However, things became more challenging when DDoS attacks were directed at the IP address without an address. In such cases, our team had to create several interfaces and distribute sites among them until we could pinpoint the attacked one and block it. This process was often accompanied by a rise in calls and emails from affected site owners, which we resolved by transferring them to other servers or advising them to wait for a week.

We also relied on a script to block IP addresses that frequently accessed the server, but this proved to be problematic when Google robots were suddenly included in the IP list. Site owners complained about their sites being inaccessible from Google's search results, but resolving the issue was not easy. We escalated the issue to higher authorities, but the response was slow, and we were instructed to manually unblock Google IP addresses upon request. Eventually, a user wrote to the general director, explaining the problem, leading to a swift resolution to the issue.

This incident occurred in 2018, and while I cannot disclose the name of the hosting company, some users may recognize it from the given image.

It is impossible to obtain the complete list of IP addresses used by search engines, making it difficult to check for them. Your best bet is to reach out to the technical support team of the search engine that seems to be causing the issue. They can help you determine if there are any problems preventing your site from getting indexed. However, if the problem is related to IP blocking, you will need to contact your hosting provider's technical support team.

Blocking IP addresses typically does not have a significant impact on website traffic, as nat, VPN, proxy, and mobile IPs are rare, isolated cases that are negligible. Instead of blocking IP addresses outright, it is often better to implement temporary blocks or add captchas (including recaptcha3) to the site. Fail2ban is not an effective solution for all websites, especially those with a large number of products and requests per page.

Resolving IP addresses is an effective way to detect search bots and block proxies, VPNs, and hosting services, without relying on whitelisting. Using test cookies and redirects can also be beneficial, as most bots do not understand JavaScript. Blocking IP ranges can also be useful, but it's important to ensure that it's done through checking captchas or custom solutions to check humanity.

It's worth noting that using footcloth in htaccess can become unwieldy when dealing with thousands of lines, making it inconvenient to remove certain IPs from the list. To improve efficiency, it's recommended to check HTTP headers for correctness and use filters for IP addresses to block "incorrect" requests like wp-login.php and others.