First Part: the Altera FPGAs have some models that allow you to store keys in volatile and non-volatile memory.
Altera explain that they use AES with 128 or 256 bits. The private key is stored in the FPGA and the external memory have the data encrypted with the public key. In the boot-up, the data in read from the external memory, decrypted with the private key inside the FPGA, and then processed to configure all the following reads (think like a config header).
Second Part: they have anti-tamper measures to prevent someone from doing exactly what you're afraid off. See: http://www.altera.com/literature/wp/wp-01111-anti-tamper.pdf
In general, it's "easy" to open the chip and brute-force extract each component position inside it, doing a kind of reverse-code attack. FPGA are programable, so an already programed and a virgin FPGA will look the same at the phisical level attack. And you end up destroying the programming when you try to open the chip and extract something from it.
And, to protect you from extracting the key, there's no function like *read_private_key* . You can write and overwrite it, but you can't extract it.