If I have a secure https://www.foo.com domain, but also have insecure http://a.foo.com and http://b.foo.com domains, is it possible to use an HSTS header with the includeSubDomains option if you only send it on https://www.foo.com?
No. We must remember though that the implementation of HSTS is left to the browser (UA, to be precise). Given the wording of the standard, it is possible that we might see some variations.
The intent is that if you add the optional "IncludeSubDomains" bit, the UA must thereafter send only HTTPS requests to subdomains too. If the subdomains aren't ready with HTTPS, I would NOT recommend that you use the IncludeSubDomains directive.
Part of the confusion stems from this statement in the standard:
From Section 5.2 HSTS Policy:
An HSTS Policy may contain an optional directive -- includeSubDomains -- specifying that this HSTS Policy also applies to any hosts whose domain names are subdomains of the Known HSTS Host's domain name.
The reference is not to the host names or fully qualified domain names (FQDN), but just domain names.
Here the definition of "domain name" is apparently critical - and I couldn't find sufficiently unambiguous reference; due to the presence of terms like "complete domain name" and "fully qualified domain name". It is possible that the standard uses the term "domain name" in the same way we use "host name" or FQDN. In that case, my answer would be wrong.
Yes. If you set includeSubDomains on www.foo.com, only www.foo.com and *.www.foo.com are accessed exclusively over HTTPS. You can have another subdomain such as insecure.foo.com domains without HSTS flag.
Compare this to setting includeSubDomains on foo.com. Now all subdomains such as anything.foo.com must be served over HTTPS.