The general rule for systems is: if you don't need it, don't install it. Perhaps we should modify that to: if you no longer need it, uninstall it.
Why? To reduce the "attack surface" available on the system - i.e. programs and utilities that can be compromised or used as part of a compromise. That's the general justification.
The specific risk for development tools is that if an attacker gains access to an unprivileged user, chances are they can then use say gcc to compile arbitrary code on your box. That said - chances are also high that you're running an interpreter such as python, or have javac around for web applications or some such - so the actual material impact might not be much different.
I personally see nothing wrong with installing the dev tools (I'd pull them in manually with yum install <tool> so they can be then yum remove'd later) and then removing them after their purpose is fulfilled. After all, all this talk of compiling abitrary code under unprivileged users actually misses the point: at that stage, somebody has already gained a foothold on your system and that is what you want to prevent.
Policy-wise, not having development tools on a webserver is in my mind a good idea - it discourages developers from "doing it live". Not that we would, mind, but well... we probably would :)