If I give someone a thumb drive I've used to copy a file from my PC to the thumb drive, can that file be traced back to my computer?
The BTK killer was caught by a similar method, although that was a floppy disk. The exact means are available in this article. Additionally, I know of computer forensics that have been used in catching theft of intellectual property when people have switched companies. In those cases, files from the source company where copied to a USB drive and then taken home.
I don't think it's an easy thing to do, but it certainly is possible. To be certain, the person doing the forensics would need access to your computer. I'd wager you wouldn't do that willingly, so that would involve a search warrant.