Disclosure of ASP.NET Server errors is primarily an information disclosure issue
When this happens you are disclosing information about the technology in use. If it's just the default errors (not the verbose ones) this is pretty limited, if it's the verbose one's you're disclosing stack traces and detailed ASP.NET version numbers. The impact of this depends on other areas of your site's security.
If everything else is perfect, there's probably not a lot of impact, however where other security issues are present the verbose errors can be very useful to the attacker. For example where an SQL Injection issue occurs, verbose errors can disclose the exact SQL statement in use, which is very useful to the attacker in exploiting the issue.
So really the exact impact is hard to say without knowing the details of your site, however given the fix is pretty trivial in most cases, I'd always recommend removing these on production sites. Setting the Deployment Mode to retail is likely the easiest way to achieve this.