First, I would think about this slightly differently. Some folks will point to Device Fingerprinting as a mechanism, but I don't think that solves the real problem here.
I'm going to assume a well hardened VPN (Multi-Factor, Certificate required etc) in this circumstance. This is essentially the standard expectation for VPN. Now this in it of itself doesn't solve the problem you describe.
Unless you have seriously strong client management including: nobody is ever an admin who shouldn't be, it's nearly impossible to prevent someone who really wants to from exporting the certificate (priv/pub) you issue for the client.
To squash these potential leaks of the certificates needed to get on VPN you need to prevent the certificate from being exported in any way. There are two ways to go about certificate management that are meant to do just that. The first is traditional Smartcard based, you can get them in credit card size, or now the new chiclet size.
Credit Card Sized
Chiclet Sized (smaller than pictured here.)
If these are untenable, or expensive (they can be both) for your organization, you can look into using Virtual Smartcards. These function essentially the same as the physical smartcards except they use the machine's TPM and a third party driver to perform those operations, and hold the relevant private keys.
Now TPM is the mechanism hardware providers will indicate is the best (and academically I agree with that logic) however, practically you'll find the way you interact with VPN clients the best is to stick to the certificate interactions. Virtual smartcards give you both the host-stickiness of the TPM and most of the time is just easier to interact with.
The reason for the SmartCard like functionality is it makes it virtually impossible (presently) to extract the certificate's private keys without a significant level of effort. Using the TPM as the mechanism now ties certificate to machine.
With good lifecycle management and tightly controlling where you issue these certificates. You can do a pretty darn good job of authenticating that devices you have issued these certificates to are the ones connecting to VPN.