The owner has the access to the machine which is equivalent (or more) of physical access to a physical machine. For example, he can clone it, event silently while it is running, and then "play" with the clone in a safe sandbox environment.
There is only a very limited defense against it, for example you can use disk encryption on the virtual machine in a configuration, where the decryption keys are only in the RAM. It still doesn't avoid the data stealing from the hosting side, but significantly hardens it: he can't "play" only with the persistent disk image, but requires also to analyze your disk encryption software to extract the key from it in runtime. In the best case, you can make this data stealing so hard as cracking a binary software image.
There are various algorithms to process encrypted data without decrypting it, but this all is only a hot research topic today.
The only for sure defense against such stealing is if the sensitive data is not on the VPS. For example, the VPS can be a datastream-level gateway to an out-of-VPS service which uses strong cryptography (including MitM protection) - if the keys are unreachable from the VPS, the hosting will have no way to decode what is going on.
You have no way to do any similar hardcore trick on Windows.