Nessus and OpenVAS appear to have fairly similar features. Why would you choose one over the other, besides the benefit of commercial support (which isn't available for Nessus Home Feed users anyways)?
A recent test of Nessus and OpenVAS shows the benefits in using multiple scanners due to the difference in the signatures:
Nessus, OpenVAS and Nexpose VS Metasploitable (blog post by Peter at HackerTarget)
Out of 15 known security holes in the system used for the test, 4 were spotted by all four tested tools (Nessus, OpenVAS, Nexpose and some Nmap scripts); 7 were only spotted by some and 4 were missed completely.
Tenable responded with an article on The Right Way To Configure Nessus For Comparison.
I think that both projects will have their strong and weak points. At the moment I'd say that Nessus appears to have a wider range of plugins available and arguably a better user interface than the standard OpenVAS client/server implementation.
In addition Nessus seems to have widened their original focus in unauthenticated vulnerability scanning to include authenticated (audit) style checking. In several scenarios this is likely to produce more accurate results than unauthenticated scanning. Having looked at these scanners authenticated scanning capabilities, Nessus does seem more advanced here than OpenVAS
EDIT : Just thought I'd add another couple of things I've noticed which could be relevant to people looking to evaluate these two. If you're looking at Database Scanning (Oracle/SQL Server) Nessus offers credentialed scanning for these, with DB level creds, which will find some vulnerabilities.
That said if you're interested in database vuln scanning NeXpose could be well worth considering, it seems to have good authenticated and unauthenticated database scanning.