I'm generating 128 bit API keys. AFAIK 128 is totally secure for generated key. In the DB I want to store hashes, not the plain text (to be protected against DB leak), but what hashing algorithm should I use? MD5 and SHA-1 are not considered secure. SHA-256 produces bigger output than the key itself so it seems like it doesn't make sense to use a 128 bit key with SHA-256.
In theory, it should be safe to use MD5 here since it is only vulnerable to collision attacks, not the much more dangerous preimage attacks which would be required to forge an API key. However, it's certainly not a bad thing to avoid MD5 on principle, since it looks bad for auditors.
Assuming the value you are hashing is randomly generated and is not a human-provided and potentially weak password, then you can use SHA-512 and truncate the output. If you want to use a standard, then there is a SHA-512/t hash defined in FIPS 180-4 in section 5.3.6 which outputs a digest of t ≤ 512 bits. If a standard is not important to you, then it's fine to just truncate the output of the hash to 128 bits.
I specify SHA-512 rather than SHA-256 because, paradoxically, it's actually faster on 64-bit systems. If your system is not 64-bit, then you will probably have better performance with SHA-256.
but what hashing key should i use? md5 or sha1 are not considered secure. sha256 produces bigger output than the key itself so it seems like it doesn't make sense to use 128bit key with sha256
It's okay for the output hash to be longer than the input being protected. Sha1 and Md5 are both deprecated for certain applications, so probably best to avoid those options.
Since you are hashing 128 bit keys you may be fine just using one round of sha256. This is based on the understanding that you are hashing random 128-bit api keys and not passwords. The advice for hashing passwords will be very different.
You can and probably should use SHA-512 (or SHA-256 if you prefer) and just use the first 128 bits of the output. There is no reason to be afraid of just throwing away part of the output.