Because it is still extension of OAuth2 Authorization Code Grant.
If the application is trusted with end user password then it can use Resource Owner Password Credentials Grant and as you say only have direct (TLS protected) call to authorization server.
In Authorization Code Grant application is not trusted with end user credentials so it have to force user to interact with authorization server directly (by performing redirect). The authorization server confirms identity of resource owner and granting access to OAuth client and then have to return the access token to the Client. In Authorization Code Grant it is done by redirect using resource owner user agent, passing the authorization code. The OAuth client then calls authorization server directly and exchanges authorization code for access token.
The two redirects are simply the way how the OAuth2 Authorization Code Grant is defined.
Now if we do not want to trust the application with client credentials then the redirects are hard to avoid - as apart of the payload that is passed there is one more "thing" handed over - the person in front of device.
The application is saying to authorization server "Here is some person, who claims to have resources managed by you, please arrange access to those resources for me."
After some time authorization server "The person you sent me confirmed your access, here is a way to obtain it."
In no place in exchange the shared identity of the person is established.
So if we get rid of first redirect, we would get "Some person claims to have resources managed by you, please arrange access to those resources for me." - clearly request impossible to fulfill.
Now for second redirect the authorization server is the initiator so it would be very hard to call back specific mobile app instance, and even in server to server scenario there is still a problem with matching response with the request.