Content sniffing. Your proposal is not enough: it will be vulnerable to content-sniffing attacks. I've written elsewhere about strategies to prevent content-sniffing attacks. There are a variety of defenses. Here are the main ones:
Content-Type: header in the response. Make sure it includes a valid MIME type (avoid invalid MIME types.)
X-Content-Type-Options: nosniff header in the response. This will turn off IE's content-sniffing algorithms, on recent versions of IE.
If you don't intend for the content to be viewed in the browser, it can help to set
Content-Disposition: attachment, to make the browser treat it as a file download.
Even these steps are not guaranteed to be enough. For instance, if the user is using IE6, they'll still be vulnerable.
(If this sounds annoying, you sure are right. Blame the Apache folks for including a crummy default configuration that broke web standards, for many years, and ignored pleas to do something about it. Unfortunately, now it is too late: we are stuck with a large deployed base of browsers that do dangerous things.)
Separate domain. A better defense is to host the user-provided content on a separate domain, which is used only for user-uploaded content. That way, a successful content-sniffing attack cannot attack your site's content. One user's upload can still attack other users' uploads, but that may be tolerable.
Check your whitelist. You seem to assume that PDF and Word files are harmless. However, those are two powerful and dangerous file formats. PDF is notorious for being a vector. Malicious PDF files abound, and can successfully penetrate many older PDF viewers. The PDF risk is so high that Chrome takes special precautions before allowing you to download and view a PDF document in your PDF viewer. Word is also a powerful and dangerous file format, that can be a host to attacks. For this reason, I would not consider Word or PDF harmless.
You may be able to redirect users to Google Docs, to view the Word/PDF file in their browser through Google Docs. Google will convert the Word/PDF file to HTML and then send it to the viewer's browser. This may or may not be acceptable in your circumstances.
Scan file uploads for viruses. I recommend that you scan all user-uploaded content for viruses, using some virus or malware scanner. For PDF files, see also How to scan a PDF for malware?. You might want to scan the upload immediately when it is uploaded. You might also consider periodically re-scanning older files (this may catch some malware that wasn't previously detected, as antivirus/malware definitions are updated).
More information. See also Mozilla's checklist for file uploads, in their secure coding standard. It's a pretty good list of good security practices.
Summary. In summary, the most powerful and effective defense you can use is to place the user-uploaded content on a separate domain. Then, as additional protection, you may want to consider the other defenses listed here.
Update: I just learned about one more problem with your scheme. Apparently, Flash ignores the Content-Type header, which could allow loading a malicious SWF, which can then do everything you'd do with a XSS. (Sigh, stupid Flash.) Unfortunately, no amount of whitelisting can stop this attack. Consequently, it appears that the only safe solution is to host the user-uploaded content on a separate domain.