There's another, indirect statement about this case:
Unless the Redirection URI is invalid, the Authorization Server
returns the Client to the Redirection URI specified in the
Authorization Request with the appropriate error and state parameters.
Other parameters SHOULD NOT be returned.
Preloading isn't as dangerous as you're making it sound. The only requirement for it to not break your site is that you have TLS working, and if TLS weren't working, then your site is unsafe anyway. The right answer is "just preload".
Currently, most URL redirection from http to https is done in error-prone ways (depending on the expertise of developers, webmasters, and hosting companies), such as using the Apache Redirect directive and/or rewrite engine. Browsers themselves cannot step in and change the scheme unless they know that HSTS applies.
One solution might be to eliminate HSTS, ...