Objectively, how effective are malware removal tools at detecting infections?

Question

I'm fully aware that concrete scientific data may not exist on this topic but I feel that the question at hand won't all boil down to opinion. Either way, with the rampant increase in sophistication of malicious software, how effective have anti-malware and antivirus vendors been at combating these threats?

To further explain, if I run a full scan with the top software currently in the industry, such as Kaspersky, Malwarebytes Anti-Malware (with the anti-rookit functionality), BitDefender, ESET NOD32, and Avira to name a few, and they all come up as clean, can I truly feel safe? Is 'undetectable' (as in through traditional mediums) malware possible?

A lot of the things I do as a hobby involve viruses, worms, trojans, and the like. There's been times where I've been worried that my main machine is compromised but all major vendors report no threats so I've trusted that blindly. I think that this may be a mistake.

Keep in mind that the question has nothing to do with the actual ability to removal said infection, just with the ability to detect it. This also isn't a self-help post but logical, non-subjective suggestions are more or less welcome.

Answer 1

The best numbers I've found so far are in the Microsoft Security Essential Report 19 where they show at page 89 that with full real-time protection the infection rate is still at about 2..5%.

Then there are numbers from the Cicso Annual Security Report 2016 where they claim a time to detection at about 20 hours, but adware and browser injections are at about 200 hours.

Is 'undetectable' (as in through traditional mediums) malware possible?

With APT (advanced persistent threats) the time to detection is weeks..years. Last average I've heard was about 200 days. So I would consider this as 'undetectable' at least for a long time. But even non-APT malware is designed to be as stealth as possible with a large amount of infections because the more persistent the infection is the better is the return of investment for the attack.

The big question is always what is really malicious. For example you could not simply declare software as malicious which reads data from the local disc and talks to the internet because this would not only include information stealing malware but also mail programs and browsers. And the less obvious the malignance is the harder it will be to detect the malware.