Online brute-force attacks are not the only threat to your password. The biggest one that comes to mind is a compromise of the password database (which happens with frustrating frequency). A strong password makes it more difficult for offline attacks on your password to fail.
Offline attacks bypass brute-force protection because they do not use the system to access your password; they have direct access to the database.
The other threat is that very simple passwords, depending on the type of simplicity and the context, are easier to "shoulder-surf" or guess. I was sitting beside a tax accountant on the train who logged into their tax system with the password of, you guessed it, 123456
. It was hard not to notice since he used a single finger and slowly pressed each key in sequence...
Also, brute-force protection doesn't help much if the password is so easy to guess that one could log in without hitting the brute-force failure threshold.
Security protection is a stack of layers of protection. You are protected from the failure of one layer by the strength of another. If all, or some, layers are weak, then you weaken the stack. So, while you might legitimately lower the cost of protection in one layer by weakening it (like making passwords that are easier for you) it means that you are trusting in the strength of all the other layers all the more. And in your scenario, you have zero control over those other layers (i.e. brute force protections). Such a calculation should not be done lightly.
You don't have to trust password managers in order to conclude that their use has less risk than alternatives. Password managers are not a magic wand. They are a single point of failure, and as software, they can have vulnerabilities. The question is: "what are you trusting instead, and is it more trustworthy than a password manager?"
For many people in many contexts, password manager weaknesses are far stronger than the weaknesses of the alternatives, like relying on one's memory or using password patterns.
Password managers are not perfect, and there are use cases where the risk analysis does not make sense. But for the majority of people and use cases, they are better than the alternatives. Don't reject password managers just because they are not perfect.
I encourage you to reassess the risks around password managers. We have a few Q&As here talking about them.