If you're wondering whether the behaviour you described can be abused, I can think of at least one exploit (described by James Kettle, from PortSwigger) that can lead to the equivalent of stored XSS, but only under certain conditions:
The value of the Origin request header gets reflected verbatim in the Access-Control-Allow-Origin response header can have ...
Okay, I found two documents that helped me answer my own question.
The page is specifically about the Content-Security-Policy header:
and it shows interesting examples of CSP.
Now on Mozilla, we have a default-src page which clearly explains that the default-src is the default for all the other ...-src options. Changing ...