Hello everyone.

The website is hosted on DigitalOcean. An email came today.

QuoteHi!

We're reaching out to let you know that your droplet centos-s-1vcpu-3gb was taken offline after it was responsible for contributing 103901.87 packets per second to a 103901.87 packets per second distributed denial of service attack. The traffic from your droplet matched a pattern of harmful activity coming from other droplets aiming at a specific target. We understand how disruptive this might be for your work; however, we had to take your droplet offline to prevent any further damage.

How you resolve this issue will depend on your use of centos-s-1vcpu-3gb-fra1-01, your technical expertise, and/or the time you have to investigate.

Option 1 - If centos-s-1vcpu-3gb-fra1 doesn't hold any data you need to keep, we recommend destroying this droplet and starting anew. This is the quickest way to restore your services. Please keep in mind that you will still be charged for your Droplet usage even if the network is down.

Option 2 - If centos-s-1vcpu-3gb-fra1 does have data you need to recover, follow our recovery steps at https://www.digitalocean.com/docs/droplets/resources/recovery-iso/ before deleting the droplet and restarting.

Option 3 - If you're confident in your technical skills and want to troubleshoot, isolate, and address the issue on your own, we offer a guide at https://www.digitalocean.com/docs/droplets/resources/ddos/ that provides some recommendations.

Let us know after you've completed your troubleshooting, and we will guide you through the next steps.

Please advise on what can be done. The sites are down, and I can't access the server either. I contacted support, but no response yet. I changed all the passwords and restored the site from a backup. I informed support about this.

What do you suggest I do?

Here's what I'd recommend next:

Assess the Damage: Since your droplet was involved in a DDoS attack, it's likely that your server was compromised. The first step is to assess whether any critical data has been lost or if there's been unauthorized access to sensitive information. If you've already restored the site from a backup, make sure it's a clean backup and that it hasn't been tampered with.

Option 1 - Start Fresh: If you don't have any critical data left on the compromised droplet, the safest and quickest solution might be to destroy the droplet and create a new one. This ensures that you're starting from a clean slate, free of any malicious software or scripts that might have been installed during the attack. It might take a bit of time to set everything up again, but it's the surest way to get rid of any lingering issues.

Option 2 - Recover and Secure: If you need to keep the data on the droplet, you should follow the recovery steps provided by DigitalOcean. This usually involves booting into a recovery ISO, which allows you to access your droplet without it being connected to the network. From there, you can check the system logs, identify suspicious files or processes, and remove them. This approach requires more technical knowledge and is more time-consuming, but it can save your data if you need it.

Option 3 - Troubleshoot and Secure: If you feel confident in your technical abilities, you can follow the guide from DigitalOcean to troubleshoot the issue. This involves identifying the root cause of the DDoS activity. Look for any unusual processes, open ports, or connections that shouldn't be there. Also, consider setting up firewalls, fail2ban, and other security measures to prevent such attacks in the future. This might also be a good time to audit your server's security policies to make sure everything is locked down as tightly as possible.

Wait for Support: Since you've already contacted support, it's also important to wait for their response. They might be able to provide more detailed information about the attack or assist you in recovering your droplet more securely. It's also possible they might offer to unblock your droplet after you've taken the necessary steps to secure it.

Additional Security Measures: After resolving the immediate issue, you should consider implementing additional security measures. This could include moving to a more secure hosting plan, setting up regular backups, or even using a managed hosting service that includes security monitoring.

Given the severity of the situation, I'd personally lean towards starting fresh with a new droplet unless you absolutely need the data from the old one. This minimizes the risk of any leftover malware or vulnerabilities affecting your site in the future.

It's also worth considering setting up monitoring tools that can alert you to any unusual activity in real-time, so you can act faster if something like this happens again. And make sure to keep everything updated, from your server software to any CMS or applications you're using.
Even though this situation is stressful, it's an opportunity to strengthen your security practices and make your site more resilient against future attacks.

So, let's say you've been compromised by some hack or a backdoor, and now you're unintentionally involved in an attack on someone else. What should you do? First of all, it's crucial to keep an eye on what's happening. It's hard to give specific advice without knowing the exact situation, but at the very least, you should check if there's still an effort to send out those malicious packets.

If that's the case, then something fishy is running, maybe some rogue scripts or backdoors on your site. Review all the logs and other data... basically, it's probably better to stick to shared hosting, especially if you're asking this kind of stuff on forums. Also, consider improving your security measures and regularly updating software to avoid such scenarios in the future.

DigitalOcean's option is quiet resonable - keep the current server as it is to investigate how it got hacked, and at the same time create a new server and deploy your sites from backups to it. You will save time becuase you wont have to wait for the old server to be fixed, but can launch the sites right away. And the old server can be deleted after the investagation - it might take one or two days or even a few hours.

How to determine why and how the server was hacked? Well, more or less as follows.

In 99.9% of cases, one of your sites got hacked and a script was uploaded to your server to conduct the attack. 0.1% chance that your admin password or even superuser (root) password was stolen or guessed. But usually, after analysis, it turns out the site was hacked - this is much simpler and often automated. Hackers run an automated process to find vulnerabilities on the site, and if any are found, the hacking script is launched. Once access is gained, the victim is added to a botnet and used for various malicious purposes by the botnet owner.

Since the server is disconnected from the network, you can't access it via ssh or the panel. However, you probably have a console (usually VNC access) that lets you access the server even when internet access is disabled. Log in to the server via the console (either through the DigitalOcean control panel or from a VNC client) and check the access logs for the server administrator accounts. If everything is clean and the logs only show your IP addresses, you can proceed to check the sites for hacking. Start by running the files through an antivirus, and see which files have been changed recently. Once you find the malicious files, you'll need to check how they ended up on your server, and then delete them or move them to a folder that's inaccessible from the internet for further study.

If old versions of site engines, their templates, or plugins are found, they should all be scheduled for updating on the new server. Then change the passwords for the sites and their databases on the new server.