I think you're making some incorrect assumptions about Java - Java itself does not listen on a port. Applications RUNNING in Java do.
So Java itself, running on your PC/server isn't listening for anything. If you're, say, running a Minecraft server, which runs in Java, then traffic to/from that Minecraft server hits both Java and the app. If the app has a vulnerability, you can be compromised. If Java itself has a vulnerability, you can be compromised by virtue of the fact that the JVM sits between network traffic and the app. I am unaware of any such exploit at this time that directly targets Java via network traffic alone (i.e. a specially crafted packet that depends on the packet only rather than a vulnerability in the Java app) - most vulnerabilities are in the ways apps interact with Java. That's not to say one does not exist - I am just unaware.
With regards to:
Regardless of how many times I change my dynamic IP, I'm getting port scans, attempts to connect to certain services such as ssh, and other phenomena.
That's a fact of life on the internet. EVERY public IP address is CONSTANTLY scanned for SSH, open ports, etc. This isn't a function of your ISP or your server - it's a function of the internet as a whole. That's why it's so crucial to have adequate firewalling in place. You're not any more targeted than any other random IP address out there.