Consider the following cases.
A vps running a selfhosted service which is listening on port 1234.
Case 1: There is firewall configured that blocks all incoming connections by default. Now port 1234 has to be explicitly opened in the firewall configuration. When a client tries to access a different port, firewall blocks it.
Case 2: There is no firewall configured. Client can access port 1234 by default. When a client tries to access a different port, kernel would find no process is listening on the port and just close the connection.
If there was a rogue process in the host, it can listen on a random port and let anyone in. But even if firewall blocks incoming connections, it can still make external connections and setup a reverse tunnel.
Only case I can think of is if a miss-configured non-malicious process happens to listen on random port and bind to all interfaces '0.0.0.0', then firewall would help block.
Is there anything else I am missing? It just appears unnecessary extra configuration for no practical benefit.
Update
Thank you for answers. I do understand firewall can do advanced things mentioned. I am trying to find out for the simple use-case. If I just have a simple VPS running some services listening on few ports. Is it better to configure a firewall and then open only the required ports, or just don't have any firewall at all.
I think the answer comes to if I can be sure there is no unintentional listener on the external interface. e.g. a service may have an api/admin port opened different from main web port. Any decent app should not listen on the external interface by default but misconfiguration do occur.