If a site uses HTTP and not HTTPS, is it useful to include a Content Security Policy? Can an adversary not just remove it?
It depends on what the attacker is able to do.
As you know, it's possible for a HTTP request / response to be intercepted, and modified by an attacker. This attacker can then alter the response to remove the CSP.
But this requires the attacker to be able to intercept HTTP requests, which is, depending on the threat model, not the easiest attack vector to get.
Many attack vectors don't require intercepting the HTTP request at all, like clickjacking and XSS injections. CSP is still an effective defense against them.