All of these algorithms, whether simple or complex, rely on well-known and often simple "hard problems". RSA for example is actually quite complex, involving numerous extensions that are necessary to make it secure, but it is still based on the simple fact that it is easier to multiply two primes than it is to factor a semiprime. Algorithms like DSA (which exploits the discrete log problem) are similar enough that an advance in integer factorization could be used to break DSA as well. AES and countless other symmetric ciphers use the substitution-permutation network. Some ciphers like Twofish use a fiestel network. Others, like ChaCha20, use add-rotate-xor (ARX). All of these designs are well-established, and a breakthrough in solving a major problem will defeat many ciphers, regardless of how simple or complex they are.
How much does complexity matter?
This means that, in the end, the complexity of a given implementation does not necessarily say much about its security. ChaCha20 for example is an extremely simple cipher, so simple and efficient that you can bet that anything faster than it will likely be less secure, yet it is built upon well-known principals (ARX). Serpent is a very complex cipher, with multiple "features" built in to prevent possible future attacks, but it, like AES, uses a substitution-permutation network. It is also extremely secure.
Cryptography and attack surface
Cryptography is not like application security where more complex inherently means more attack surface and greater bugs. Software is traditionally built on the assumption that any bug can break the expectations of the software completely. Ciphers and other cryptographic primitives, on the other hand, are often designed with the assumption that guarantees of the time may not hold up in the future as new classes of attacks are discovered. Serpent was created with this in mind, adding not only extra features, but also increasing the number of rounds.
The world of software security and the world of cryptographic security are vastly different, and established laws in one do not necessarily apply to the other. This includes such important mottos as "complexity is the enemy of security". The only similarity between software security and cryptographic security is that crypto is typically implemented in software. Implementations may be more insecure as they get more complex (compare libnss to libsodium), but the algorithms behind them do not follow the same rules.
In addition, some examples which you claimed as being more complex and thus potentially less secure are actually the most secure. It is based on formally proven mathematical models, not just assumed hardness problems. Regardless of development of new attacks, it is thus always possible for GCM to promise an acceptable minimum level of integrity for a given secure cipher. It being complex says nothing about something as irrelevant as the security of universal hashes over binary Galois fields (which GHASH, used by GCM, is based on).
Use only what you need
If you're trying to decide how many primitives a cryptosystem needs, you're already doing it wrong. In cryptography, you should only use what you absolutely need. Not because extra complexity makes you more vulnerable, but because a system that is designed well does not need more than the situation calls for. This is why cryptographers design ciphers with typically only 128 to 256 bits, rather than something crazy like a megabit. Is a keyspace of 2^10^6 bigger than 2^256? Absolutely, but anyone proposing seriously using or designing a megabit cipher would be laughed at. The same applies to primitives used in cryptosystems. Use what you need, since using extra only demonstrates a lack of understanding of the fundamentals of cryptography.
For example, if you have a need to authenticate data, you need to use authenticated crypto, like ChaCha20-poly1305 or AES-GCM, or an HMAC. If you have no need for authentication (say, because you only care about confidentiality of data-at-rest and nothing more), then don't use authentication. If you don't need it, GCM or poly1305 will only make your implementation more bloated and slower. If you do need it, then absolutely do not forgo it for simplicity. Hell, if you only need to encrypt a single block, and with a unique key, using ECB as the mode of operation is totally fine!
But what do you need? (Obligatory warning)
If you're asking yourself this for a serious project, as opposed to just for education or fun, stop. Use a popular crypto library. The choice of primitives depends on many factors. While the hardness that is the core of each primitive is undisputed, the precise implementation details lead to a lot of gotchas. DSA for example is quite secure, relying on the hardness of the discrete log problem, but it requires a random value, k, for each signature, and if k is in any way predictable or repeated, the algorithm fails catastrophically, revealing your private key. EdDSA on the other hand is based on the same principal, but it is entirely deterministic and requires no randomness to perform signing operations.
If you don't know what you need, here are a few "hard to go wrong" options. If you want to use current and secure algorithms. These are all very fast, and quite secure:
ChaCha20 for encryption. It is uses add-rotate-xor internally and is very fast in software.
Poly1305 for authentication. It has a proven security model and can be plugged in to any symmetric cipher.
x25519 for key exchange. It uses ECDH over curve25519, and uses nothing-up-my-sleeve values (modulo p = 2^255+19).
Ed25519 for signatures. It boasts quite a few security benefits such as side-channel resistance. It uses EdDSA with curve25519.
BLAKE2 for hashing. It's not particularly special, but it's secure and is based on well-regarded security features of the SHA2 family.
ChaCha20-based CSPRNG for randomness. As long as you have a strong seed, any plaintext you throw at it will give you cryptographically secure randomness.
If, on the other hand, you need to use NIST crypto (for example due to regulations, because your marketing department really wants to use the term "military grade", or just because you don't like DJB):
AES for encryption. It has been around for a long time and shows no signs of going away. Fast hardware implementations are common.
GCM for authentication. Like poly1305, it has a proven security model, though it's a bit tricky to implement safely (another reason why you should use a crypto library).
NIST P-256 for key exchange. While it uses sketchy values internally, it's just about the only strong, standardized key exchange technique.
RSA for signatures. It is old and well established.
SHA2 for hashing. Use whatever member of the SHA2 family that provides the necessary digest size or collision resistance that you need.
SP 800-90a revision 1 for randomness. It includes a set of DRBGs (Deterministic Random Bit Generators), of which Hash_DRBG and HMAC_DRBG are considered the least controversial. Revision 1 simply removes the NSA backdoored Dual_EC_DRBG from the list.
If you simply need to use a library, go with libsodium. It is popular and uses plenty of DJB crypto, and is designed to be easy to use. Crytography is hard, and is really the only field where appeal to authority is a virtue, not a vice.