Is it really that simple?
I guess having a master password will prevent that, but the drawback is that it is asked each time I open the web browser.
Information Security Stack Exchange is a question and answer site for information security professionals. It only takes a minute to sign up.
Sign up to join this communityWelcome to the real world! The master password on Firefox passwords is not different than the lock on the front door of your home: if you use it to prevent others to enter without permission, you will have to consistently unlock it to enter your home and lock it when you go out.
There is no magic here: you have to decide what protections you use and if possible ensure that they are relevant according to the expected risk and threat. If you think that the threat level is very low, you could even decide that the physical lock on your front door is enough to protect your computer and its content. And it is not a joke, because systems not connected to any network and only used to control mechanical automates need no special security beyond the physical access. And I must admit that my personal computer only has a trivial password because I know that if a attacker puts its hands on it, leaking photos of my familly will be be the least problem. But my password vault has a strong password because leaking it would give access to my bank accounts.
So it is up to you to decide what level of threat and risk you want to accept. But if you have no master password on Firefox, yet store passwords in it, anybody having an access to the config is able to steal all of them along with the site they are related.