Any web page you load will have your IP address
In order for your browser to download the content associated with a website, your computer will send requests which include your IP address (this is how the data knows where to be sent). However, your antivirus software may have prevented the connection. Depending on how your AV works, it may have prevented you from making a connection to the suspicious website, and your IP address would not be known to the suspicious website.
It is unlikely someone has your home address from your IP address
whois protocol could be used to determine a physical address from an IP address. However, in home-user applications, your ISP's information will be returned, not your own. Furthermore, ISPs often dynamically assign IP addresses to their clients, so the IP address you use today may not be the IP address you use tomorrow.
How else could an attacker get your home address from your IP address?
An ISP could store information on the modem such as a customer account identifier which could lead to an attacker determining your home address if they compromised the modem. If an attacker compromised your router, they could sniff traffic to look for your address traversing the network unencrypted or attempt to correlate a Wi-Fi router's MAC address or broadcast name with a Wi-Fi geoloation database such as WiGLE. If an attacker could compromise a computer on the local network, the attacker may be able to find documents which contain the user's home address. Keeping the modem, router and computers well configured and up to date will mitigate the likelihood of this happening.
Assume your IP address is known
You can't operate on the Internet without exposing your IP address, so you should assume that it is known. Additionally, (assuming IPv4) there is a relatively small number of IP addresses available, which means scanners may be trying to connect to your IP address even if you have never "given" it to them somehow.
A well managed local network will mitigate the risks of an attacker having your IP address
Because you must assume that your IP address is known or will be guessed, you should set up your network to protect your computer.
- Keep your router up to date with the latest firmware, and check its configuration.
- Connect your computers to the router and the router to the Internet. This will give each of your computers a private IP address that is not routable from the public Internet. Your router will then forward requests from all clients using the same public IP address.
- For each of your computers, set up a firewall to block access that is initiated from the public Internet.
Be wary of all software, including VPNs
You should be wary of any software you download, especially those offering free services. If you are using a free service, it is likely that your data is what is "paying" for that service. If you want to maintain privacy on the web consider using Privacy Badger.