In addition to the accepted answer of the authentication provider being often a different server from the resource provider (thus having HTTPS required over the resource provider which might not), even in the case where they are both the same server with HTTPS it is better to have a short-lived access token and long-lived refresh token.
Think about it: how many places handle the refresh token? On the client or browser, it should be in a central place that looks up the current access token and if that has expired or the request fails, uses the refresh token instead (typically if your service provides an SDK, this is one small subset of that SDK which can be heavily reviewed for potential security vulnerabilities). On the authentication provider, it is only ever handled in the code path which handles the refresh_token grant type. If you want to prove that the refresh token is never sent to a log file or sent to an external repo, it is a pretty small and manageable collection of code you need to look through to verify that.
Conversely, the access token is used hopefully only in the one place on the client (again), but everywhere on the resource provider. Every HTTP request coming into your app has the header in the Authorization header. If you want to prove that the access token is never put into a log file, and that no one has put in malicious code to export your access tokens off to https://myaccesstokenfarm.example.com, that is a very large (bordering on impossible in some codebases) task.
The general security principle here is reduction of attack surface. Refresh tokens could be pulled from a man-in-the-middle attack just like an access token could be, but by restricting the attack surface to just one URL on one server and with just one executing code path, it is much easier to do everything in your power to make that particular resource secure.
As an arbitrary example, if you have HTTPS compression turned on, you might be vulnerable to BREACH attacks throughout your API; but you want compression on for efficiency, so you live with that possibility. On your authorization endpoint, however, you turn compression off and take the extra steps to ensure there are multiple layers of defenses against known attacks.