I want to encrypt a few files on my computer (not FDE). To do this, I'd like to use something that requires having some external usb thing with the key on it plugged into the computer. Price is not really an issue for the usb device (ideally in the hundreds of dollars). Any recommendations for solving this problem?
It depends on what security features you are looking for; specifically, how much you mistrust your own computer.
If you just need the decryption key to reside on a USB device, then you can put a key file on a basic USB flash drive, and use GnuPG. The key will of course make it to the main computer, because the computer will do the actual decryption work. But at least the key will not be written to a file on the computer; it will be in RAM only. This is the cheap method. It works in the case where you do not trust your computer when it is powered down; e.g. this is a laptop and you fear that it could be stolen while you are not using it.
To go one step further, you may want to consider encrypted drives like these ones. The encryption and decryption keys are known to the device only, and all encryption and decryption occur in the device, not the host computer. Some models can even be unlocked with fingerprints (a reader is embedded in the device). If the host computer is hostile (e.g. compromised with a nasty virus), it can still plunder your data when the device is inserted and unlocked, but at least you have some control on when such things may happen.
If you have really sensitive data that must never be shown to a potentially compromised computer, then the most natural solution is to buy a dedicated computer, kept offline, for this data.