There are a number of defenses you can use to help prevent and recover from theft.
The first thing you should look into is full-disk encryption, e.g. LUKS, TrueCrypt, or PGP. This will prevent an attacker from reading any data on the disk, even if they steal the hardware. You will need to enter the password at boot, though, so for unattended remote hardware this might be problematic unless you have access to lights-out management (e.g. HP iLO or Dell DRAC).
On top of this, you should ensure several other mechanisms are in place:
- Strong physical security in your data center (e.g. locks, biometrics, CCTV, alarms)
- Security procedures should be put in place at the data center. All people entering should be made to sign in, and all hardware access / changes should be logged and signed for.
- Good server racks come with appropriate fixtures for padlocking servers in place. If available, this feature should be used. Select a strong padlock that is resistant to bolt croppers and shivs.
- BIOS administrative password set, to prevent the boot order being changed.
- BIOS boot password set, if possible (may require physical attendance at boot)
- Application credentials can be stored in a dedicated HSM to help prevent recovery of data in the event of theft.
- Epoxy resin can be used to disable physical ports, to prevent unauthorised devices from being plugged into the system.
- Asset IDs should be properly set in the BIOS or server management console, and should be logged in an assets registry.
- UV ink should be used to tag all devices. You can buy UV security pens very cheaply, and they're very useful for property identification in the case of theft. It is often worth marking individual hard disks as well as the server chassis.
- Tamper-evident mechanisms, e.g. security tape, can be used to ensure that the hardware has not been tampered with. A quick and easy way to use it is to tape up the server and sign the tape with a marker pen. If the server is opened later, the seal will be broken. New tape applied over the top will not carry the same signature.