My current understanding is as follows:
The client (a web application running in the user's browser) generates a nonce, puts it into the browser's session storage, and redirects to the authentication server, passing the nonce as parameter. After authentication, the authentication server redirects back to the client, passing a signed id token containing the nonce, which the client must validate against the nonce from the session storage.
If the nonce is not validated, an attacker could substitute another id token, by tricking the user into visiting his site, and sending a redirect with a valid id token obtained from a different request.
In addition to the id token, the authentication server also replies with an access token. If the nonce is not validated, an attacker could substitute a different access token, by tricking the user onto his site, and redirecting with that token. This might be used for a client side denial of service attack (in particular if browser tabs share the token storage, i.e. use local rather than session storage).
In summary, nonce validation is necessary to trust the id token. If the id token need not be trusted, nonce validation may be skipped.