Currently working on a website which allows users to upload data files for automated processing. For both security and user guidance purposes, we implemented an extension whitelist both client and server side to only accept common data suffixes like .txt and .csv and so on.
We have now discovered that some data files which we get sent don't have any file extension at all.
These files are not executed by the processes after upload: they are parsed for text patterns. So having unverified files up on the server does not - theoretically - present a big security risk. The data being processed, however, is personal data covered by data protection law, so a breach would be catastrophic.
Is it safe to allow users to upload files without extensions is this instance? And, as a wider question, is it ever good practice to allow a site to permit these files?