Yes. This is not only possible, but this is an existing feature offered by
Microsoft (Note: Microsoft's system is likely breakable in theory, since TPMs and theoretically possible to compromise). "Shielded VMs" are designed specifically to be hosted remotely while still secured against "compromised or malicious administrators in the fabric, such as storage admins, backup admins, etc." They accomplish this by encrypting disk/state of the VMs.
See A closer look at shielded VMs in Windows Server 2016 for discussion (Server 2019 adds Linux support).
You should be suspicious of the giant asterisk next to "Server administrator" in the infographic below. High-level, securing a VM from the server administrator requires the use key-based or TPM-based attestation.
Well, let me offer a theoretical solution to this problem which is secure. Of course, this is only useful for theoretical purposes, but it suggests that your problem is, at least in principal, solvable:
You have the following "stuff":
1. Key: A private key.
2. Plaintext: Your private data
3. Server: An untrusted machine (untrusted because someone else has physical access)
To update the contents of the server:
1. Encrypt the entire plaintext with your key.
2. Update the encrypted plaintext to the server.
To retrieve the contents of the server:
1. Download the entire cyphertext from the server.
2. Decrypt the cyphertext with your key.
I believe this approach is provable safe, on condition that you're willing to make various assumptions (i.e., the same assumptions you make when using SSL and the like).
So, it's possible, at least in theory, to store secure stuff on an untrusted machine. To go a step further, people have even found ways (currently impractical) to execute encrypted CPU instructions on an untrusted machine.
Therefore, I believe that it is theoretically possible to boot an encrypted server remotely. The next question, of course, is if there is any sort of practical, provably safe solution. I'm not sure. However, even if the answer is no, it might not be no forever.