Regularly, unidentified bots visit my site, and it's unclear why they're there. They may be collecting data or attempting to push my site down in search engine rankings. Perhaps a bot-tracking service is trying to pressure me into paying for their services.

Regardless, dealing with them was once easy, but after switching to a new hosting service, I no longer have access to an IP address tracking system. While I use the CIDRAM WordPress plugin to manage 80% of these bots, I still face some obstacles due to poor instructions. My goal is to ban these bots by name, as I suspect they come from a single source.
Do you have any suggestions for how to do so effectively?

Quote from: SIROTA on Aug 03, 2022, 11:52 AMHow to ban bots by name?

Just hang on Cloudflare and block the subnets you want in a couple of clicks.
It's free.

Quote from: Ali_Pro on Aug 03, 2022, 12:10 PMJust hang on Cloudflare and block the subnets you want in a couple of clicks.
It's free.

Connected, now in the mail, which is on the domain, letters have stopped coming.

What needs to be screwed up where?

Quote from: SIROTA on Aug 03, 2022, 01:34 PMWhat needs to be screwed up where?

mx records in DNS settings in CF.
Add.

https://www.cloudflare.com/learning/dns/dns-records/dns-mx-record/

https://developers.cloudflare.com/dns/manage-dns-records/how-to/email-records/

The fact of the matter is that the records have already been updated and Cloudflare has stopped sending letters through the connected domain. I turned off the proxy for email records in the DNS settings, and still the letters do not reach. Letters go to SMTP, but they do not come from IMAP.

---

Post Merge: Aug 03, 2022, 05:26 PM

---

In NX - MX is slightly confused)
Now everything is fine.

Great, why no one says that Cloudflare successfully keeps bots out of the site, and you don't need to pay anything.
I pressed only 2 buttons (check JavaScript and something else) and all the bots fell behind, 4 bots left, and to hell with them. I used to suffer, track down, set up SIDRAM, ban subnets along with real users. And it turns out everything is simple and even my participation is not required.

Thank you kind man for your advice!

This issue has been discussed on the forums for many years.

It is much more efficient to leave only the necessary countries, and start the rest on js check + for the necessary countries, add to js check the subnet of hosters that are googled.

Some bots are tested, then you can just hang a captcha, they will not break, it will be expensive for them.

To prevent bots, parsers, and spam from visiting your website, you can limit their access using .htaccess file rules. While using robots.txt for this purpose is recommended, it's often ignored. There are three methods to block unwanted bots: by IP address, User-Agent, or IP mask (by country). However, blocking bots by IP address can be tedious and ineffective since IP addresses change frequently.

A better solution is to ban them using their User-Agent name regardless of their IP address. You can also ban bots from specific countries where servers for SEO services, spam bots, and other parsers are located. There are various unwanted bots, the most popular of which include AhrefsBot, SemrushBot, MJ12bot, Riddler, and others. To block them via User-Agent, you can add certain code to .htaccess. Alternatively, you can also forbid access by IP address mask, which can be achieved by adding specific lines to .htaccess.

Here are some steps that might help you effectively ban these bots by name while using the CIDRAM plugin:

First, you need to identify the user agents that the bots are using. This information can usually be found in your server logs or by using a plugin that tracks visitor activity. Once you have the user agents, you can create a blocklist in the CIDRAM plugin.

Head over to your CIDRAM settings and look for the option to add custom rules. You'll want to add the user agents you identified earlier in the correct format. Make sure you follow the syntax closely, otherwise the rules won't work as intended.

If you still have access to any logging or monitoring tools, try to see if you can correlate the bots' activity with the user agents. Some bots may change their user agents frequently, so keep an eye out for any new patterns that might emerge.

Another thing to consider is utilizing a .htaccess file if your server uses Apache. This file allows you to set up bans based on user agents as well. You can add something like this:

RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} (botname1|botname2|botname3) [NC]
RewriteRule .* - [F,L]
Just replace "botname1", "botname2", etc., with the actual names of the bots. It's an additional step but can provide another layer of protection.

In addition, ensure that you are keeping your WordPress and all associated plugins up to date. Sometimes, security vulnerabilities can be the gateway for bots, so patching these frequently is essential.
Don't hesitate to reach out to your hosting provider. They might provide some insights or tools to help you track those pesky bots more effectively. They might also have their own firewall or protection systems that can help manage bot traffic better.