What are the (possible) privacy concerns in using a TPM chip ?
Can my keys or my datas be leaked to government agencies or criminals ?
Can my OS be infected using the firmware in the chip as an attack vector ?
The TPM compromising the OS is pretty unlikely. The chip talks to the host computer via a low-speed serial bus comparable to a serial port; the code handling data received on that port is much simpler than let's say USB-related code, so if someone wanted to own the machine they would rather use USB-based solutions (especially since you don't even have to exploit any vulnerabilities in USB-handling code as USB has a built-in vulnerability : it trusts any USB keyboard right away).
Now, whether the TPM is good at what it's supposed to be doing is a different issue. Personally I don't trust TPMs for full-disk encryption as the key is transmitted unencrypted over the LPC bus, so for someone with physical access to a machine (even turned off) all he has to do to own it is to connect a logic analyser to the LPC bus and grab the key from there. Once that's done the attacker defeated the FDE even though the machine was off, thanks to the TPM.
As far as other uses of the TPM where it generates and holds a secret key, it comes down to whether you trust the manufacturer to not have put in a secret command that would make the TPM spit out the key it was supposed to protect (or a "bug" that would allow running arbitrary code on the TPM), as well as trusting the manufacturer to have sufficiently hardened the chip against hardware attacks.