Note: this somewhat elaborate answer won't go down with everyone as the one downvote already shows, but I'll leave it here, because I stand by it. Sure, I don't just say one or the other, not black and white, but unless a moderator or the community feels this answer doesn't belong here, it'll stay no matter how many downvotes it gathers. Heck, perhaps I can break the record for the most downvoted answer after all? :)
There are many many angles to this question and I won't be able to cover all.
Up front a partial disclosure, I am working in the AV industry, but I won't give any more details. Another disclosure, I haven't been using any "realtime" AV mechanisms on my private machines in the past ten or so years. However, I have run offline scans on my machines from time to time.
Several angles to look at the problem
Drive-by-infections and trojans are but two of the potential attack vectors, so careful browsing will not completely relieve you of using common sense or doing your homework in other areas (keeping your software patched and such). There are many more, including targeted (usually known as "spear-fishing", "social engineering") attacks.
Underestimated potential for attacks
For example it has been shown by researchers that placing a USB key that looks like a flash drive but behaves like storage + mouse + keyboard can be used to get into otherwise highly secured areas. No one thinks of an attack first when finding a USB flash drive, right? But it could be.
Anyway, there are many more attack vectors on a wider scale. Most common examples include:
- vulnerabilities in software you use
- The various browser engines
- Simple rendering of strings can exploit vulnerabilities (found in the past in font rendering)
- Java when used as a plugin (although not really necessary any more these days unless you visit a lot of websites from the 1990s ;))
- use of a number of complex and/or proprietary file formats (pictures, documents ...) where the software decoding the file format has errors.
- turning your own brain off at the wrong moment
- getting an email from a loved one with an attachment? Uh oh, email senders can be faked like postcards. You can use cryptographically signed mails to guard against this.
- actually going to some "erotic" site and just to see that one image or video executing that "downloader"
- Firmware can be "trojanized" so that your OS doesn't get the slightest chance to detect any wrongdoing on part of the computer.
- Bluepill by Joanna Rutkowska was a similar approach in that it uses virtualization instructions to move the running system into the guest position while the rootkit itself retains the position as hypervisor.
To just name a few.
Btw: the inverse is also true. One could also engineer a mouse or a keyboard that includes storage and triggers the execution of something off that storage. Here's a project that makes this accessible to a wider audience.
But let's look at the quote you gave.
To be blunt: I refused to install any kind of antivirus or personal
firewall software on most of my computers (but see Update 1/1/2012,
below.) This included a Windows XP Home system that was used by my
children as a web surfing / email / game system. I suffered zero
infections during this time.
That clearly shows that the quoted person is not very knowledgeable about the subject. No one, including AV vendors can tell you that you aren't infected with 100% certainty. It's a silly claim. Just because there is no sign of infection doesn't mean there is no infection. There are very few methods, usually involving live CDs/DVDs (example Qubes) which will protect against most attack vectors, but one can usually come up with a scenario where they fail. The most practical are sandboxes that separate programs from each other and this is the approach that Qubes uses.
What I'm saying is that the person uses the wrong justifications and makes the wrong conclusions - not that there is anything wrong with not using AV programs.
AVs won't protect you necessarily, especially pure AVs
My question is, are there any documented studies that either prove or
disprove these claims of being able to browse the internet carefully
and having the same amount of risk of infection as that of browsing
the internet with anti-virus software?
Again, browsing isn't the only attack vector through which malware can enter your system (the autorun mechanism in Windows together with a vulnerability in the icon handler for link files allowed Stuxnet to spread). It was as easy as plugging a flash drive into the computer. The next thing is that a pure AV solution will not filter the websites you access and will only trigger whenever something hits the disk (can also be the browser cache, though). So this makes it clear that a pure AV solution doesn't provide a very good protection against browsing-related infections in the first place.
However, we can assume that most malware needs to persist on the users' machines to be useful to its author - unlike in the past when malware authors proved their knowledge through the creation of viruses and such, these days malware writing is very much a commercial endeavor - such as skimming banking data or using the GPU for Bitcoin mining and what not.
So when it hits the disk, the AV can catch it. Provided that the AV knows about it. And make no mistake, most AV vendors have long departed from the overcome signature-based detection or only use them as a supplementary detection method. That's contrary to popular belief that I have seen voiced time and time again. I am only aware of a single vendor that is basically sticking with the old ways of signature-based detections. The majority of vendors uses a detection that will catch usually hundreds or thousands or related malware samples at once. Still, the AV company has to have come across the malware in order to detect it.
It's a race and in the end also an arms race. Malware writers are using Virustotal and other sites in order to find out whether their creations get detected. Since there is money to be made, be it directly or by renting out your infected machine as a drone, there is enough incentive to keep a low profile.
Dispelling a myth ...
Just for those believers in AVs the results you see in many, if not most, of the certifications are severely skewed. That has to do with two things:
- some of them want payment and that of course creates and obstacle for disgruntling the AV vendors
- most of the tests allow for loopholes that are mercilessly abused in favor of test results to show off in the respective marketing reports of the vendors
In my opinion one of the better tests is the RAP (Reactive and Proactive) test which measures how good AVs are doing with "outdated" virus signatures. But there are certainly more things that would have to be improved in order to make these tests more objective.
Conclusion: you can't buy security
AVs will provide an additional layer of security to the layman, even to the expert. Period. But - and that's a big but - this added security can never offset lack of education in security-aware computer use. Conversely I hold that, yes, if you are paranoid and diligent enough you don't need an AV as in a must have. The technological layer of protection will not offset the protection you gain from due diligence, paranoia and awareness.
The security provided by an AV can very much be compared to airport security. There's a lot of show, but there's also some actual added security involved. But don't believe that the AV will protect you - or others for that matter - from getting infected.
In my opinion the best protection is awareness and diligence. But unfortunately there is nothing like a driver's license required for computer use. So failing that the next best protection would be sandboxes, whitelisting approaches and also AVs and other anti-malware solutions. They'll never be able to give you a 100% protection (well, whitelisting might), but they can protect you from a wide range of prolific threats. So when in doubt, sure go for an AV solution. But that's where it gets difficult. They differ in quality and even the tests - as I pointed out above - are often severely skewed. Adding another AV engine into the mix may help but there's no guarantee. If inside a single product, the integration of several engines will likely be good and not add too much overhead. However, if you install several AV products in parallel you are asking for trouble.
keep in mind: Strictly speaking we can never tell whether a
system is clean, whereas we can state that we didn't find
anything. Every claim to the contrary is either Marketingese or is
going to obsolete the AV industry as a whole quickly.
(source: my answer over on SuperUser here)
A yet better security net are backups and frequent rollbacks to a known-clean state - but that's just in my opinion.
Commercialized malware doesn't need privileged access
Over on SuperUser I wrote this:
One more thing. Many Linux (also MacOSX) users fall prey to the
assumption that malware that doesn't have privileged access can't hurt
you (which is also often cited as a reason why there "isn't" malware
for unixoid systems, which also isn't entirely true). This couldn't be
further from the truth. Although this prevents it from establishing a
system-wide stronghold, it won't keep malware from skimming data from
your personal files etc. A rogue browser extension installed in your
own profile will still be as dangerous to your account as the one that
does it to all accounts. If you do your internet banking with the
rogue extension installed, it makes no difference whether you are