Strictly viewed, it's not correct. It's not the CA that generates the public and private keys, but the requester. There's no reason the CA should have access to the private key. The private key should be kept private (though it's needed to establish every new secure connection, so it must be also on-line).
There's no point in keeping the public key hidden. By publishing the public keys, the CAs can increase the confidence level in the certificate itself, by giving another path to verify the identity. The purpose of public key cryptography is that there's no practical way to compute/impersonate the private key with only the public key accessible.
And, despite that, public keys/certificates are not published in all cases - for example, in the case of client certificates, assigned for user email addresses/persons, it might even be dangerous to publish all issued certificates.
On the other hand, publishing server certificates has the advantage that every other entity running servers can verify and confirm that noone else has issued a certificate to someone else for their name - this is actually a subject of a recent initiative called Certificate Transparency.
See - https://www.certificate-transparency.org/known-logs
Update: I agree that the quoted description is quite vague, and somewhat confusing.
Perhaps the poster misunderstood the same way, that the CA publishes any public key that it issues a certificate for. But it's not what it's about. What it appears to say, that when the CA initially creates its own certificate, with the public and private keys, it is then sharing that specific initial public key (well, not really, but the whole CA certificate) with the world "at large". This is the whole the point of a CA certificate - so that everyone finding a certificate issued by this specific CA will be able to verify it with a public key already available, in a way that no third parties can maliciously (at least not easily) interfere with.
