The critical difference is in what is kept secret.
Take RSA as an example. The core principle of RSA is simple mathematics. Anyone with a little mathematical knowledge can figure out how RSA works functionally (the math is almost half a millenium old). It takes more imagination and experience to figure out how you could leverage that for security, but it has been done independently at least twice (by Rivest, Shamir and Adleman, and a few years before by Clifford Cocks). If you design something like RSA and keep it secret, there's a good chance that someone else will be clever enough to figure it out.
On the other hand, a private key is generated at random. When done correctly, random generation ensures that it is impossible to reconstruct the secret with humanly available computing power. No amount of cleverness will allow anyone to reconstruct a secret string of random bits, because that string has no structure to intuit.
Cryptographic algorithms are invented out of cleverness, with largely-shared goals (protect some data, implement the algorithm inexpensively, …). There's a good chance that clever people will converge onto the same algorithm. On the other hand, random strings of secret bits are plentiful, and by definition people won't come up with the same random string¹. So if you design your own algorithm, there's a good chance that your neighbor will design the same. And if you share your algorithm with your buddy and then want to communicate privately from him, you'll need a new algorithm. But if you generate a secret key, it'll be distinct from your neighbor's, and your buddy's. There's definitely potential value in keeping a random key secret, which is not the case for keeping an algorithm secret.
A secondary point about key secrecy is that it can be measured. With a good random generator, if you generate a random n-bit string and keep it secret, there is a probability of 1/2^n that someone else will find it in one try. If you design an algorithm, the risk that someone else will figure it out cannot be measured.
RSA private keys aren't a simple random string — they do have some structure, being a pair of prime numbers. However the amount of entropy — the number of possible RSA keys of a certain size — is large enough to make one practically unguessable. (As for RSA keys being practically impossible to reconstruct from a public key and a bunch of plaintexts and ciphertexts, that's something we can't prove mathematically, but we believe to be the case because lots of clever people have tried and failed. But that's another story.)
Of course this generalizes to any cryptographic algorithm. Keep random strings secret. Publish clever designs.
This isn't to say that everything that isn't a random bunch of bits should be kept secret. Kerckhoff's principle doesn't say that — it says that the security of the design should not rely on the security of the design. While cryptographic algorithms are best published (and you should wait a decade or so before using them to see if enough people have failed to break them), there are other security measures that are best kept secret, in particular security measures that require active probing to figure out. For example, some firewall rules can fall into this category; however a firewall that doesn't offer protection against an attacker who knows the rules would be useless, since eventually someone will figure them out.
¹ While this is not true mathematically speaking, you literally can bet on it.