Write a program that causes an immediate shutdown if someone inserts something into a USB port when you have not first entered a password to white list that device. Make sure you use a limited account so nothing serious can be done on the computer without admin password (like the Linux command to dump contents of RAM).
Use newer DDR3 RAM as it "forgets" things within a millisecond when power is removed. For added effect somehow lock the tower case, and perhaps install a button which would cause a shutdown when opened.
Full disk encryption, make sure you use Linux.
From the book Computer Forensics for Dummies it stupidly appears to be standard practice with desktop computers to pull the power cable, and make an image of the HDD before attempting any forensics. If you followed the above steps then they will be unable to recover anything due to encryption.
Note the above is not true with servers, where advice is to shutdown normally.
Optional extra: add a script to clear RAM on clean shutdown (what Tails OS uses). Although this is not necessary if you use new DDR3 RAM, it might help you sleep better.
Forgot to mention that in some extreme cases they could freeze your RAM with liquid air which would cause it to take hours to "forget" stuff.