The attack surface is smaller with the hardware PIN pad. The secure channel between the smart card driver and the card can't protect against snooping over USB between the keyboard and the computer, nor can it protect the key presses from being read when they reach the OS before going to the smart card driver and then the secure channel. Both of these threats fall under the key logger attacks you mentioned, and both are mitigated strongly by hardware PIN entry because there's no pluggable USB path to monitor, nor is there a complex kernel and driver infrastructure handling each key press. Some hardware PIN devices even have tamper-evident housing to reveal if someone tried to implant monitoring in the pad itself.
Regardless of how the PIN reaches the card, there is still the threat of malicious challenges being signed by the card after the PIN has unlocked the signing function. Aside from using air-gap security, one way to detect malicious behavior is knowing how many things you're intending to sign (possibly with an audit log) and matching the expected signatures against the counter on the card, which is like an odometer for signing operations.
A hardware PIN pad can also make malicious signing requests obvious because an attacker would have to make your card sign their data instead of yours, which would then require another signature to happen for your data. Hardware PIN pads help here by always requiring PIN entry for each signature. If you enter your PIN, something gets signed, and it's not the thing you needed signed, something fishy may be going on. While you can configure something like GnuPG to require a PIN for each signature or challenge, you're at risk of an attacker re-enabling PIN caching to sign both the malicious data and your own with one PIN entry.
Devices like the YubiKey 4 have a lighter version of this protection where you have to tap the card for each signature. This still provides a good way to detect someone maliciously having their data signed instead of yours (because you'd tap, and something would get signed, but your data would not yet be signed).
I don't think the secure channel is enough to prevent malicious signatures from being created because it only closes the attack between the driver and the card, while something outside that system has to initiate the signature process. Malware on the same OS would still be able to request signatures.
On the other hand, without the secure channel, an attacker might be able to monitor signatures over the USB interface to the reader, but that's the same attack that could capture a PIN entry from a standard keyboard and then directly request signatures from the card. The PIN capture attack is the easier one, too. I can buy a key logger off Amazon.com, but I doubt I'd be easily able to get my hands on a pre-made tool to intercept smart card signing traffic. Once I have the PIN, I could run a USB host connected to the card reader via something like a Raspberry Pi, and I'd be able to sign away the next time the card gets inserted.
In summary, a hardware PIN pad provides effective protection against malicious PIN capture as well as easy detection of malicious signing requests (via one PIN entry = one signature). Barring air-gap security and heavy audit procedures, that's the best you can do.
