But the whole point of using password managers (yes, that's another thing) is that you don't have to write down all the passwords...
Backup codes are distinct from passwords.
Passwords are something you need every time you use the account. The danger of writing down passwords is they will naturally be near or even on your computer. Anyone with physical access to your computer now has your passwords.
Backup codes are for emergencies and, ideally, are never used. They can be stored securely and well away from your computer.
And, perhaps most importantly, targeted, physical attacks are a very low risk for most people. If you're a business, or don't trust the people who have physical access to your stuff, a basic safe might be warranted and useful for lots of things. But for most people Joe Burglar is not going to be searching for your backup codes, they'll just take your whole computer.
Why do I need to keep my backup codes for each platform on paper?
You don't. You just need them somewhere offline.
You could put them on an encrypted USB thumb drive. Omit your usernames if you feel you can remember them. Store the key in your password manager. Now an attacker must have three pieces: the drive, the key, and your usernames. Put the drive in a safe location (not necessarily a safe) away from your computer. It can be in the same room, just not an obvious location. A "bug out" bag is a good choice, in the event of a disaster you will be able to restore access.
You can also put the backup codes into your password manager along with your One-Time Passwords. This does put all your eggs into one basket, but that's the trade off for using a password manager.
Finally, you can use a dedicated device such as YubiKey.
Is there any way I can be sure to regain access to my account that's been setup to use 2FA, when I lost my phone, without having to fallback to silly backup codes...?
Depends on the site. Some will allow you to recover by sending you an email.
Security is about managing trade offs and risks. Preventing attacks opens opportunities for new attacks. So long as those new attacks are more difficult you're headed in the right direction.
Weak, reused, and written passwords are solved by long, unique, random passwords in a password manager; a password manager brings its own much smaller risks. Multi-Factor Authentication further secures your accounts, even if your password is compromised the MFA protects you and informs you someone has your password, but now the risk is losing your MFA device. This is solved with backup codes. The backup codes could be stolen, but now the attack must be physical, they must come to your location, which immensely reduces the risk.