If someone has rooted your box, then I think they can get your passwords from either method with not too much effort. They will be able to get your password file for the password manager and the password to that file via a keylogger. If you just use remembered passwords, they will collect your passwords over time as you key them in.
If someone has physical access to your machine to install and retrieve a keylogger, then they can log on to your machine if they have physical access for significant time. If they don't have enough time for that, then you would be safer with the password manager as they don't have your password file (only the password to it). If they do have time to get root access to your box and enable remote access to use at their leisure, then you are in the same position as in the first paragraph.
When you think about it like this, there is very little if anything that memorization holds over password managers. And that you and your passwords are hosed if someone nefarious has physical access OR can root your machine remotely. Preventing these two attacks is key.
You first need good physical security to prevent physical attacks. Then you need to prevent remote attacks - implement good network perimeter security, enable a minimum of services and have good security practices for the ones that you have no other choice but to enable. And also practice safe browsing habits, keep your machine regularly updated etc. i.e. the preventative measures that prevent you getting rooted in the first place. And you need to have backups in case you do get rooted. If you do get rooted, then you should assume that you need to change every password whether or not you use a password manager.
About the only thing memorization buys you IMO is perhaps a bit of time before someone logs your important passwords. You sacrifice a lot for this. Memorization is not scalable. The more sites you use (and as the web permeates our lives, need for more usernames and passwords grows), the more things you need passwords for. And they increasingly benefit from being strong. To make them strong requires a lot of effort on the user's part, along with making memorization difficult. Or if you write them down on a sheet of paper, you have the same problem as the password manager, except that it's harder to backup and sync, more conspicuous, easier to misplace, in plain text, etc. And you have to type them in all the time, rather than copy and paste.
At least using a password manager gives you the ability to have very strong and different passwords to every site on the internet you use (which may be many). And also the ability to easily remember the different usernames, along with notes about the specific sites. If you memorize a strong password to your password manager, they have to be able to get your keystrokes somehow to make use of your file, or they need your file to make use of your keystrokes.