I think that returning the code to the user-agent is not secure
Why not? The code is completely useless on its own (without the client secret or PKCE code verifier) if the PKCE extension is used, as is now recommended for all user-facing flows. The only threat is if an external party (attacker) obtains the code and is able to spoof to the client that the attacker is in fact the user (or at least their browser/other user agent), which PKCE mitigates since the code will be associated with a different code_verifier/code_challenge pair. Since the code is (supposed to be) single-use and very short-lived besides, the window for doing this is extremely short even without PKCE. It is also possible in most cases to avoid even exposing the code in a URL (which is slightly insecure, in that they are often logged and may briefly appear in a URL bar) by getting the user agent (browser) to POST the code to the redirect URL rather than doing a GET request, if that's your concern, but again, PKCE renders that consideration moot anyhow.
Besides that, the basic idea as described isn't very practical. Suppose Alice and Bob are both users on client Foo, and they (through Foo) make authorization requests to authorization server Bar. Bar verifies the identities and access authorization of Alice and Bob, and then... tells Foo that Alice and Bob have both authorized it to access their accounts. Great, what is Foo supposed to do with that? Foo doesn't know which of its users to let use Alice's access token, or Bob's access token!
It's possible to work around this problem. If the authorization server includes the user's "state" token (as it currently does in the redirect) in the direct request to the client, and the state is uniquely identifying for a user, then this information would allow Foo (the client) to correctly associate each request from the server with its own user. However, in other cases (such as when the client is not a uniquely addressable service, but is instead e.g. a static webapp or server-less mobile app on the user's device), it's not even possible for the authorization server (Bar) to communicate with the client (Foo) except through the user agent. Similarly, if the state isn't used (it's not essential in every context, though it's recommended in all of them) or isn't stored on the server in a useful way, this wouldn't work.
I don't actually know why OAuth (which is generally a somewhat "over-engineered" protocol family, in that it tries to have the best solution to every possible use case, at the cost of considerable complexity) does not in fact permit this approach where it would be viable (client has a known, reachable endpoint address and includes a uniquely identifying and unpredictable token with every authorization request). As far as I can tell, it has no major security weaknesses. While PKCE does mitigate even the minute threat of an attacker capturing the authorization code before it's used and submitting it to the client as their own, I'm not convinced that the mitigation adds less complexity or more security than the approach above.
However, I have some guesses. First of all, as noted above, PKCE basically solves the problem already, and if it requires some additional complexity on client and server, well, so does this approach. This proposed modification requires a bigger change in the protocol; fundamentally modifying the data flows, rather than simply adding one step to each of client-side and server-side plus an additional parameter to each request and a new error response. PKCE is also usable in more cases (doesn't require a client addressable without routing through the user agent) and thus has been adopted even for clients that can't store client_secret values securely (which used to be limited to the weaker Implicit flow, and mostly wouldn't be able to use this modification). This modification also adds the novel requirement that the authorization server make outbound requests; in the current flows, the authorization server need only listen for and respond to inbound requests (from the user agent and the client); that has security implications for hardening the authorization server against attacks. The modification also adds nothing if the user agent is in fact compromised (e.g. by a malicious browser extension) since that's functionally equivalent to the world where the code was stolen by the attacker and used to impersonate the user to the client.
Given all that, this would functionally be an entirely new OAuth flow/grant from the existing ones. Such a thing could be created and standardized, but I'm not sure it adds any value beyond the Authorization Code + PKCE flow, which is already standardized and widely adopted.