I've set up a site on Digital Ocean without a domain yet, so there is only the IP. Despite telling no-one of its existence or advertising it, I get hundreds of notices from fail2ban that various IP's are trying to hack my SSL port or are looking for PHP files.
But how do they know that I do exist? Where do they get the IP from?
Pretty much what @DeerHunter said. It's trivial to scan the entire internet. If they want, they can target all-known digital ocean droplets that are online.
They can do this on a timer so that when you go offline, or online, it will just keep trying as those may be high-value targets that could become vulnerable at a moment's notice.
Let me give you a very rough coding example. Let's pretend your IP address is 104.16.25.255. Let's get the IP address of www.digitalocean.com so we can easily check for associated IP addresses. www.digitalocean.com returns 104.16.25.4. Let's scan everything: 104.16.25.*
Let's assume we want to try and find all nearby associated IP addreses. Assume programs can handle numbers and patterns very well. Here's an example of an integer being incremented:
i++;
This increments the current value of i by 1. Let's assume i starts off as 1. After i++, you'll get 2. Check out this painfully simple loop:
for (int i = 1; i < 256; i++)
{
scanIpAddress("104.16.25." + i);
}
An alternative one-line bash variant would be as follows:
for ip in `seq 1 255`; do scan_thingy_command 192.168.0.$ip --options -oG lol.txt; done
You just scanned 104.16.25.1, and changed i from 0 to 1. As the whole loop continues, it will go from 104.16.25.0 to 104.16.25.255. I don't have time to scan and look right now, however, it's possible that this tiny block doesn't just belong to digitalocean.
To find more targets on DigitalOcean, a programmer may change the numbers even more. For example, introduce another loop that nests the aforementioned loop on the inside, and add j: scanIpAddress("104.16." + j + "." + i);. This will allow them to scan 104.16.1-255.1-255.
From there, they can keep going backwards and nesting for loops until they get the entire internet. There are other, more efficient ways to do this, such as masscan, but this is the most basic way.
Again, this could also be done on the command line with one line:
for oct1 in `seq 1 255`; do for oct2 in `seq 1 255`; do for oct3 in `seq 1 255`; do for oct4 in `seq 1 255`; do scan $oct1.$oct2.$oct3.$oct4 --stuff; done; done; done; done
The above example was a really rough example. They may be doing more, their code might be different, and they may be using entirely different methods and/or programs. However, the concept is pretty much the same.
It's also possible that the programs in question are just targeting everyone en masse.
If it's online, whatever you are hiding, they will find it... or try to find it.
However, depending on your web server, you can try http access controls such as .htaccess. If you're using access controls - again, this depends on your web server - then it's likely that you'll be able to prevent others from viewing/accessing pages.
That won't protect you against non-website login attempts, though. And if you're denying them access to non-existent webpages, they now know you're really online, and can focus their attacks more easily! However, it's still good practice.
Here's an example .htaccess deny for Apache (2.4 and later):
Require ip 192.168.1.100
In the above example, you're denying everyone access to that folder, except your IP address. Keep in mind, 192.168.1.100 is a local IP address. You'll have to replace that with your public IP address.
Also, keep in mind that if your attacker is running a proxy/VPN on your machine, they can still access those pages. If your attacker already has access to the website, they can either edit the .htaccess or remove it. Nothing's 100%.
Just don't put anything online if you aren't ready to be scanned. Everyone has a plan until they get port-scanned in the mouth.
AS14061 Digital Ocean, Inc..
The IPv4 address space is limited to only 4,294,967,296 addresses.[note 1] Given enough bandwidth, it becomes trivial to scan every single IP address out there, especially if you're the owner of a botnet consisting of thousands of hacked devices.
With IPv6[note 2], things are a bit more tricky: with over 300,000,000,000,000,000,000,000,000,000,000,000,000 addresses, it becomes impractical to enumerate them all. However, there are still various means by which the addresses can be discovered; for example, in a recent case Internet-of-Things search engine Shodan was caught using NTP servers to discover new IPv6 hosts when they synchronized their clocks.
The gist of this: if you're not ready to be probed, you shouldn't be on the internet. Scanning the entire IPv4 internet can be done in a matter of days, and your IPv6 address will get discovered as well – unless you are not using it at all.
Note 1: Some of these are not available because they have been reserved for special purposes.
Note 2: IPv6 is only available for Digital Ocean users when they have enabled it.
But how do they know that I do exist?
They don't know that you exist. They don't know they're talking to you: they just know they're talking to a computer with a particular IP address. IP addresses are a lot like phone numbers. If you dial a legitimate area code followed by a random number with the right number of digits, there's a decent chance you'll get to speak to somebody, especially if the area code has many subscribers. That doesn't mean you "knew that person exists": it just means that you found out that their phone number is connected.
Where do they get the IP from?
They don't get it from anywhere. It's just a 32-bit number and there aren't really so many of those. 134.183.96.2 there, that's an IP address. I got it by mashing my keyboard (and deleting invalid digits). It probably belongs to somebody because almost all IP addresses do. The bad guys just systematically scan the address space; they'll find any IP address because "finding" just means "generating enough valid numbers".
But how do they know that I do exist?
They know that something exists on that IP address because their scanner is telling them so. Most likely they didn't come looking for you, they just stumbled across that IP address by scanning large chunks, or the entire internet.
http://blog.erratasec.com/2013/09/masscan-entire-internet-in-3-minutes.html
So you can scan all IPs from your home PC in about a year, although your ISP might notice and ask you what you're trying to pull. Some blocks are more likely to contain poorly protected servers, specifically the blocks assigned to hosting providers. Scanning only these blocks is very cheap, fast, and easy, and the likelihood of success is comparatively high.
Scanning from a single IP is problematic, because your IP will get blocked soon, so these people use botnets, or the commercial equivalent: cloud services. You don't need to build your own botnet. Instead you can rent a botnet.
Most likely they don't know that the given IP belongs to you in particular. But they do know the IP pool which belongs to your hosting company and they use that information to scan for active hosts. You can use iptables to improve the security of your box, or to limit the number of connection and connections types but other than that there is nothing else you can do. The internet was meant to be used to share information. If you don't want to share just disconnect the box (development box). If it is a development box it would be a good idea to move it locally and provide no access to it outside of your LAN.
