It depends on how much protection you need the HSM to provide. While Smartcard-HSM (which the Nitrokey HSM is an implementation of) will provide encryption operations on the dongle without releasing the key, it only supports external programmability from what I could find.
The problem with this is that while the HSM may protect the keys themselves from compromise, it can't protect the key from usage if authentication information is provided. If using a software authentication via the host, it would be potentially compromised via keyloggers or actions could be submitted by a hack when you next authenticate to the device.
A programmable HSM, on the other hand, will allow for more of the allowable encryption operations to be performed in card, including defining what operations should be performed and under what circumstances in a more secure manor. This provides an extra level of security as you can put further protections around the usage of a key that make it harder to abuse, however programmable HSMs are much more expensive devices.