Given a valid access token alone (you don't know username+password), should you be able to get a refresh token?
The idea of refresh Tokens is you use them only once - to get a new refresh token and a new Auth token. This way the token to go on was never used before and can only be used once.
The Authorization Server serves them as a response to the authorization grant, if requested by type. This step can be skipped and you can directly request Auth Tokens. But Auth Tokens are not supposed to be exchanged for other grants.