Typical USB tokens (Nitrokey, YubiKey...) allow an everyday user to store PGP keys and use them to encrypt email, harddrives and so on.
The same vendors also offer distinct products called HSMs (Nitrokey HSM, YubiHSM). The suggested use case is protecting CA certificates. They also have a USB token form factor.
Why is a different hardware product required for this use case?
Hardware Security Modules's (HSM's) are used in systems where a company has determined that the risk involved requires a higher level of control that a personal key.
The Use of HSM's for Certificate Authorities
HSM's are common for CA applications, typically when a company is running there own internal CA and they need to protect the root CA Private Key, and when RAs need to generate, store, and handle asymmetric key pairs.
Uses outside of a CA
HSM's are suggested for a companies Privileged Access Security System. These system's use the concept of a "vault" which is unlocked via a key or multiple keys. These keys are usually only required on startup of the vault.
With vaulting systems, typically options for storage are:
In secure systems, this allows key to be generated without a human needing access to it, stored in a system that is FIPS Level 2+ compliant, and only accessed when a system starts.
Using an USB Key vs a HSM
Primarily, end user USB's are designed for the end-users access. They are storing keys which might authenticate them, have encryption keys for data they are responsible for, or are part of a process that identifies the user is who they say they are.
Examples of this are the use of a YubiKey to hold the private encryption key for sensitive company data. You might have access to a trade secret, or are allowed to perform analysis on compliance critical data such as PII and Healthcare or Credit Card Data.
HSM's are not designed to be a personal authentication unit. They are installed on servers to provide a secure generation, storage, auditing, and distribution of keys for high risk applications and systems.
Examples include the storage of server certificates for authentication between systems, encryption keys which are used by processes but don't need to be seen by humans, and shared keys between servers used to access encrypted data or establish encrypted connections.
Pseudo-Summary
You could think of the use cases in terms of people verses systems.
If a company has a need to provide increased security of systems which a person uses require access to as part of there job, something like a YubiKey makes sense.
If a company has a need to provide increased security to systems where processes and the systems themselves need access but a user wouldn't normally be required to know, then a HSM is the better option.
Anecdote
Imagine walking around with they key to your companies entire certificate authority in your pocket that provides the authenticity for 1200 servers and that Key slipped out a hole in your pants. Now imagine the nightmare of having to re-issue a new CA certificate, all the new signing certificates, and all the server certificates.
edits:
At the suggestion of @MikeOunsworth, I updated and expanded this answer.
[This is speculation based on vendor website -- I have no direct experience with these devices]
My speculation is that a specialized HSM version of the device is not required, but since it is designed for this specific use-case, it will have more features tailored to this use-case. Much like how you don't need a hammer to drive in nails; you can use the handle of a screw driver.
Taking the Nitrokey product comparison sheet as an example, we can see that the end-user devices support a wide range of end-user applications like PGP and 2FA / OTP, a built-in password manager, encrypted USB storage on the device, etc. Meanwhile, the HSM only supports cryptographic keypairs, can hold many more keypairs, and has specific features for managing PKI/CA keys.
In a nutshell, HSMs provide features over personal USB token such as:
YubiSecure?R Künnemann gives the following description:In the second part of the paper, we analyze the security of the protocol with respect to an adversary that has temporary access to the authentication server. To address this scenario, Yubico offers a small Hardware Security Module (HSM) called the YubiHSM, intended to protect keys even in the event of server compromise.