Since a perfectly honest network card acts under the control of the host CPU, a malicious hardware element in the CPU can perfectly betray the user's secrets. After all, when you have some malware in your machine, the malware is just software, and it runs on the CPU -- and yet your secret data can escape.
Any piece of hardware with DMA access can read and write memory at will, and thus can modify OS and application code as it gets executed. This is sufficient to do everything that kernel-level malware can do, i.e. quite a lot of evilness. This includes the CPU itself, the GPU, the hard disk controllers, the network cards, the USB controllers...
Even if you are intent on having a dormant backdoor, which does nothing until activated from the outside, it can still be done purely in the CPU, or in the DMA controller: it suffices to have the CPU or the DMA controller inspect memory transfers and wait for a specific pattern, which triggers the hostile code. Attacker then just has to send a ping request or some other packet with the said pattern: the DMA controller will see it when it transfers it from the NIC to the main RAM, and the CPU will see it when it recomputes the IP packet checksum.
People are afraid of hardware backdoors because they feel they have no control over it -- and that's entirely true.