# Using file extension and MIME type (as output by file -i -b) combination to determine unsafe files?

We allow users to upload a number of files, all of which we either send over to scribd (doc, xls, ppts, etc) or display as a video ourselves (flv, mov, mp4, etc in flowplayer).

To avoid users uploading unsafe files, we check against a set of known "safe" file extensions and then check the output of the file -i -b command which gives us the MIME type.

Usage: file [OPTION]... [FILE]...
Determine file type of FILEs.
...
-i, --mime                 output mime type strings


Is this adequate protection to keep 'unsafe scripts' off our server or do folks use something different?

Warning. The steps you describe are not enough to be safe, if those files are available from your servers.

Explanation. Because browsers do content-sniffing in a variety of circumstances to guess at the appropriate MIME type, there are a variety of subtle cross-site scripting attacks that remain possible. The general category is sometimes known as content-sniffing XSS.

See the following research paper:

The following blog posts and messages also discuss this threat:

Defenses. I recommend that you adopt the following defenses and mitigations:

1. Host the content on a separate domain, which is used only for hosting user-uploaded content. This will sandbox it, so content-sniffing XSS attacks can only attack other users' content, and cannot attack your site. (For example, Wikipedia and Facebook use this defense.)

2. Make sure to set a correct Content-Type: header on every response that serves user-uploaded content. Vet the content-type to make sure it is one of several options on a whitelist of safe content types. Avoid sending an invalid MIME type (e.g., */*, unknown/unknown, application/unknown), and avoid responses that lack a Content-Type: header; those are subject to browser content-type sniffing and thus at high risk of attack. Avoid the MIME types that cause IE7 to do content-type sniffing (see the Barth et al paper for a list, in Table 4).

3. When serving user-supplied content, never serve one with an unsafe MIME type that can trigger execution of active code. For example, avoid all of the following: text/html, application/x-shockwave-flash, as they can contain privileged code. Unfortunately I don't think it is safe to serve user-supplied Flash content from your site.

4. When serving user-supplied content, include a X-Content-Type-Options: nosniff header along with the Content-Type: header, to disable content-type sniffing on some versions of IE.

5. For content that you don't expect to be displayable in the browser, mark it so the browser will handle it as a file download: e.g., add a Content-Disposition: attachment header.

(If this sounds annoying, you sure are right. Blame the Apache folks for including a crummy default configuration that broke web standards, for many years, and ignored pleas to do something about it. Unfortunately, now it is too late: we are stuck with a large deployed base of browsers that do dangerous things.)

• "Because browsers do content-sniffing" My gut feeling is that you should serve sniffable content, test if the content has been sniffed, then block access from these hopelessly broken browsers. (But my gut feelings may not be very practical.) – curiousguy Nov 4 '11 at 14:41
• @curiousguy, yup, you can do that if you don't mind blocking a good chunk of your users from seeing your website. However, most sites don't want to turn away even a fraction of their users. – D.W. Nov 4 '11 at 15:41
• Unfortunately I don't think it is safe to serve user-supplied Flash content from your site <-- What about from usercontent.mydomain.com? – user5575 Jul 9 '12 at 1:33
• @acidzombie24, as far as I know, that should be fine, as long as you don't mind that one user's uploaded Flash content can attack another user's uploaded Flash content ... and as long as you don't trust usercontent.mydomain.com in your crossdomain.xml files. If you want to protect users' uploaded content from each other, you could have one subdomain per user: alice.usercontent.mydomain.com, bob.usercontent.mydomain.com, etc. – D.W. Jul 9 '12 at 6:43
• @d.W. It doesn't only apply to file uploads right? In fact, every dynamic page which modifies itself based on text content from users (wiki) could be hacked.... in this case, since we are not embedding images or files, Defense(1) wouldn't work right? – Pacerier Jul 13 '12 at 1:50

Probably not safe. There are various attacks that trick the content type sniffer into believing a file is the wrong content type. Here's a particularly infamous example: