Creating a secure password generator is fairly easy, provided you have two components:
A cryptographically secure RNG. This means the output of the RNG is a bit stream that must not be predictable under any conditions. In most cases however, you can just use a PRNG designed specifically for key generation like /dev/urandom, /dev/random, or CryptGenRandom. Regular PRNG like Mersenne Twister should not be used for key generation. You should not attempt to write your own CSPRNG unless you have a deep background in CSPRNG mathematics. This is the part where most "regular people developing password generators" often don't really pay enough attention to.
A one-to-one encoding algorithm. This algorithm encodes the bit stream produced by the RNG into something acceptable by the system that needs to use the password and, for memorizable password, the preference of the user. A one-to-one encoding is a function that does not have any output collisions for its inputs, so a the function does not lose any entropy produced by the RNG during the encoding. This is the part that is actually extremely easy, but most "regular people developing password generators" often overcomplicate unnecessarily, and end up weakening their password generation method.
Example of one-to-one encoding algorithm: Diceware, base64, hex encoding, any perfect hash function. Example of transformation algorithm not suitable for password generation: most regular hash algorithm.
Once you have these two components, all you need to do is ask the CSPRNG for a bit stream of length n, where n is the required strength of your password, and encode that bitstream using your selected encoding algorithm.
Pitfall: You should parameterize the encoding algorithm so it can never produce a password that would be rejected by the target system's password policy. If you simply reject a password and generate it again using different random value, you are reducing the entropy. In theory, this does mean that you will need to know the exact password policy detail of each system you need to generate for to correctly calculate your password strength. In practice, you can often simply add a few more bit strength to compensate for the entropy loss due to such overcomplicated password policy.
Pitfall: For memorizable passwords, the user must be trained not to reject any password that they end up finding hard to memorize as this also reduces the entropy. This means that memorizable password encoder should be designed as much as possible so that it produces passwords/paraphrases that's most likely to be accepted by the user in the first try. There are few encoding algorithms that attempts this, for example EFF Diceware wordlist or Grammatical Diceware.