One assumes that you want to actually use the ILO capability, so "don't plug it into the network" isn't a useful answer. However, this is one of the few areas where it may make sense to have a separate "management" network. Put all the ILO interfaces onto a single network and limit that network's access, either via routing, firewall, or ACLs, so that only those trusted administrators machines can access the ILO. You should rarely need the ILO addresses to be able to go generally outbound, as the whole purpose of an ILO card is just to allow you to get "in front of the machine" without running down to the server room (which can be hours away in many cases).
As far as the firmware goes, it is reasonable for you to update all the firmware yourself. This not only guarantees an outsider can't slip malicious firmware past you, but it also may ensure that any security updates that apply to that hardware are in place. As illustration, the following link shows how to update and/or reset various ILO firmware:
(Illustration only, I don't doubt that there are 17 proper locations describing how to handle 9 different Sun ILO platforms. And of course other vendors have their own ILO equivalents which would be handled differently in each case!)
P.S. ILO was originally HP's designation for lights out management, I could have sworn Sun used a different nomenclature, but I can't pull it up at the moment. However, like many people, I long ago began to use "ILO" to refer to "Whatever the vendor's called their remote access card thingy," so I'll just call it all ILO here.