I want the hard disk where my data is stored to be secure from foreign external access. Is there a physical possibility to protect my hard disk or any kind of data storage, like a computer architecture where my memory is physically disconnected from the whole system as long as I want to access it?
What you are asking about for your storage is often called Air Gap. You can build it quite easily in a number of ways. But there are plenty of drawbacks:
Whenever you want to do something with your data, you will have to connect your Memory. Anyone attacking you will simply wait for that moment. Stuxnet seems to have worked that way. Computer viruses generally know how to cope with this since the beginning, when they used to spread via floppy disks. I have not checked in detail, but I guess recent ransomware is hibernating some time on the victims computers to make sure that enough relevant devices are mounted (Neal Stephenson even wrote a novel around that plot: Reamde).
Additionally, if you need to work with the data on a regular basis, mounting and dismounting the device will be quite burdensome. So you, or whoever is supposed to use the system might get bored and stop disconnecting.
Still want to do it?
As mentioned by others, you can use a thumb drive, external hard drive, SD-card, etc. If you want to defend your data against destruction (e.g. ransomware), a device with write protection (e.g. SD Cards) might be preferable.
By the way, you can also physically disconnect an (additional) internal hard drive.
If you are not so strict about the disconnecting being physical, you can simply
unmountyour drive (please check the specific steps needed on your OS). A casual attacker might miss an unmounted drive, but I would not rely on that.
Last, but not least, you can put your data on an encrypted partition (e.g. using VeraCrypt or BitLocker). Unmount the partition, when not in use. Actually, while this is not exactly what you asked for, it probably will be your best choice in most scenarios, as it protects against whole a number of attacks. You can put that encrypted partition on an external drive, if you still want your physical separation. Also, be sure to back it up frequently to prevent destruction by ransomware.