As a rule of thumb, if it looks like spam then it is spam. Automated tools can be defeated, but a human brain (mine, in this case) is harder. Antispam software is useful to weed out most spams automatically, reducing the problem to a size where human filtering is tolerable. It is in fact relatively easy to identify dangerous spam because in order to be dangerous, the spam must have some hook: a clickable link to a funky-looking domain, an attached executable file (or Zip archive containing an executable)... The ambiguous spams are thus the spams (or non-spams) which are mostly harmless.
For instance I once received a spam whose complete body was pure text (no HTML version, no attached file, just ASCII text) and consisted of a single word: "Theravada". This is the name of a branch of Buddhism so the only potential effect of that spam might have been to help me reach enlightenment, which I don't categorize as dangerous.
The problem, here, is that human filtering of emails requires a brain who is well aware of how, technically, a given email can bring harm to a machine, so this works only for me or any other InfoSec specialist, not for generic users. (Also, don't read your emails while drunk.) This also emphasizes the point that antispam filters are never absolute, especially since they rely on heuristics to which spammers continuously adapt; at best, good rules will help reducing the amount of spams to human-manageable totals.
In a generic site, a workable compromise may be the following:
- Use "normal" tools like SpamAssassin to block the 90% of "obvious spams".
- Use whitelists to automatically deliver emails coming from known good sources. This raises the question of whether a "known good source" could be impersonated; DKIM can help (i.e. if an email is guaranteed, through DKIM, to come from a given server and that server has been whitelisted, then let the email go through).
- Let pass emails which are "obviously" harmless. This is relative to how well the users are trained; a pure-ASCII email can still ask for the user to send his password to firstname.lastname@example.org, and if your users will fall for it then about no email can be deemed "obviously harmless".
- All the remaining emails can be accumulated in a quarantine zone, to be inspected by knowledgeable humans.
Human inspection of emails can have legal ramifications with regards to the expectation of privacy of communications; even in a business context you still have to make sure that all relevant policies and contractual clauses are "legally clean".
There are many companies out there who make a pretty penny selling and/or operating antispam systems, so we might infer that utterly defeating spam is probably not an easy thing to do.