It circumvents every other security measure such as ... 2FA
It doesn't and anywhere that it seems to doesn't have 2FA. Any site with an even half-decent 2FA implementation - even the weak SMS-based approaches - should require you to provide the second factor when resetting the password, or at least not log you in (after resetting) until you provide the second factor. This is a basic minimal requirement for any system that claims to use 2FA: there is no flow ending in an authenticated state where the user didn't supply two factors.
OTOH, lots of sites still don't support 2FA, while all the major webmail systems do (and some non-webmail, though mostly those are corporate ones). Your email account should be one of the most secure you have (because it's usable as an authentication factor everywhere that has a "Forgot Password" or "Magic Link" option). Of course, people are bad at security (e.g. some of them try to call things 2FA when they allow logging in using only the single factor of an email account...) but as a user, you can and should ensure that your email account is one of your most secure accounts.
As for abandoning "Forgot Password?" flows, people forget their passwords constantly. If you do this, people will either just not use your site, bombard your support staff with requests for help logging in, or use extremely weak and reliably guessable passwords (similar to what happens when you force people to change their password periodically). None of that is good (well, not for you, at least).