I recently wrote this answer, in which I explained the process of HSTS preloading. However, I noticed that I didn't actually know the exact mechanism for fetching preload information works. I have checked the Chromium Documentation, MDN Web Docs, hstspreload.org and others, but they all only explain what purpose preloading has, but not the technical details.
As such, I came up with two possible ways it could work:
- Browsers regularly download the full preload list - This would be quite wasteful, depending on the size of the preload list. Given that roughly a million domains are registered every day, this list would continuously grow and become unmaintainable, especially given the fact that the vast majority of sites won't ever be visited by every individual user.
- Browsers query HSTS status ad-hoc - This would be a simpler approach. If a browser wanted to go to
example.com, and no HSTS information would be available, the preload list would be queried about whether or not HSTS preloading was enabled for that site. This would have several downsides, as the preload list would now have very heavy traffic, and would also have privacy implications for each user.
So my question now is: How is it actually implemented? And are the downsides mitigated in any way?
Some documentation points out that there is a list, which is distributed together with the browser. This list for Chromium can be seen here, and it includes a lot of high-profile domains, like Twitter, PayPal, etc...
However, I have reason to believe that this list is not the "full" preload list. First of all, the list only contains a bit more than 500 non-Google domains, which seems rather small. Secondly, according to hstspreload.org, the domain
mechmk1.me is currently preloaded and has been so for several years already. However, this entry is absent from the list distributed together with Chromium.
As such, I believe that Chromium actually has two lists: One that is shipped together with the browser, that contains the most "important" domains, and one for the less important domains (anything that is submitted to hstspreload.org, but not present in that list).
My question is specifically about the latter list, i.e. not the list that is distributed together with the browser.