It will be always possible. For example, if there is a shell script, it doesn't matter how hardly you avoided its execution, a simple sh something
will probably execute it.
If it is out of the virtualhost root directory, it is surely a good idea, it closes out the most common attack (uploading something which isn't what it looks, and then tricking the webserver to execute it).
Note: On Unixes, the file extension doesn't mean too much for the OS to determine, what it has to do with it, the OS decides it mainly based on its content. There aren't really even a file extensions (they are simply part of the file name, after the first point).
If you want to make sure:
- First, don't use "negative lists", i.e. don't specify what "shouldn't be done". By default, everything should be forbidden. Have a - short - positive list, what "could be a done".
- To apply this conception, I think the best idea if you examine the contents of the files. For example, if you know that they should be pictures, then check their content, and allow only png, jpeg, ...etc files.
- In the case of HTTP-based file upload, you could also insert constraints to the mime type of the uploaded files, but don't forget: it is set by the client side, i.e. they can say an
application/png
mime type, while it is essentially a php script.
If an uploaded content doesn't match your short list of the allowed file types, then give an at least partially misleading error message to the user (for example, "upload not allowed" or similar), and initiate a security alarm.