Currently, the YubiKey can be setup by the user during configuration so that it is write protected. This is so the device cannot be compromised by some sort of malicious script being loaded onto it, it's also to prevent the integrity of the key being compromised as well. Although, if someone has the physical device, then really there is no need to go any further in compromising the device - at least in the sense of the device's static authentication, since you have the device itself.
But, in short, the YubiKey has measures to prevent malicious code from being loaded back onto the device. But this has to be setup by the user during initial configuration of the keys that are stored on the device. If someone were to grab ahold of the YubiKey prior to the end-user setting it up, then in theory, yes it could be compromised.