TL;DR:
- An encrypted database/file with a user specific password as the encryption key.
- Generally yes (depending on the security of the app);
Longer version: When answering such a question you need to consider:
- Whom are you protecting from;
- The risks of breakage;
- Has this problem been already solved by others;
I do not think, that you are protecting from the user him/her-self (as others seem to assume). You are protecting from non-legitimate, abusers of the application w/ malicious intent.
Risks: well, if they keys are stolen, then the abuser can grab data, steal accounts, compromise the data or even the public image of the user.
The classic way to solve this is authentication/authorization (based on memorized strings of data ("What the user knows") or biometrics ("Users properties") or ownership ("what the user has", the 2nd part of 2FA)) by which you differentiate legitimate users from illegitimate ones.
As for Has this problem been already solved by others - how do you think password managers like keepassX do it? KeepassX is a local database of passwords, key files, other sensitive strings of data - essentially your problem.
They use are 3 ways (and combinations of some of them): password, key, or a tie to existing authentication measures (like windows account). Depending on how secure you want your implementation to be - pick and choose. If you choose only password (which I would in most cases do), hash it with a cryptographic has function, like sha512 + salt or bcrypt so even if a malicious actor knows the hash, he has to do work to guess the original string. Using this password to encrypt the key will provide the necessary protection.
I do not advise to use only windows credentials, since it is not a good idea not to ask the customer for a password when launching the app if it is a security sensitive app (again, this is how keepassX does it).