I have 2 domains, one where the website is hosted (lets call it www.a.com), and another (lets call it www.b.com) which has a CNAME record pointing to the first one (www.a.com). Which one should have a SSL certificate? Should they both?
Certificate validation is done against the hostname given in the URL, which means you'll need a certificate for any hostname which you expect to be used inside a URL.
Thus, if you want to use both
www.b.com in the URL you need a certificate for each, even if they are the same host and if one redirects to the other. DNS settings like same IP address or alias (CNAME) do not matter at all, all that matters is the name given in the URL.
Just to add to the existing answer, if you want to secure multiple domains using a single certificate, you can explore Subject Alternate Name (SAN) certs, which allow you to protect more than one FQDNs. Quoting from the example given in the linked article:
With a Multi-Domain (SAN) Certificate, you can secure:
This makes Multi-Domain (SAN) Certificates perfect for any environment where you need to secure multiple names across different domains
From your question it seems like you want your client to be able to connect to both
https://www.b.com so SAN cert would be one way of doing it.
It is clear that the certificate authority will issue certificate for fully qualify domain name and validate your domain name, which you want to use for website address. In that case, if you wish to continue with two domain names (www.a.com and www.b.com), then you have two choices as below.
1. Get individual certificates for each
You can purchase different two individual certificates for each domain. As a result, the certificate cost would be high and management process will become unbearable for you.
2. Single certificate for multiple websites
It is good to go with SAN certificate that is able to secure multiple domains with a single SSL certificate and it is available at reasonable price. SAN also offers simple certificate management, as you have to manage a single certificate. Moreover, you can add other domains anytime if you require.