Kerberos itself is generally safe to use over the Internet. It was, after all, designed to be secure over one of the world's most hostile public networks: the MIT campus network.
A Kerberos whitepaper, "Why is Kerberos a credible security solution?" addresses all of the points you raised. To quote in part:
A password that is never disclosed or sent over a network is much more difficult for an attacker to purloin. Consequently, Kerberos authentication of users does not require that passwords be presented to the authentication service. Instead, the Kerberos authentication service uses cryptographic protocols whereby the user can prove possession of a password without actually revealing it.
In a distributed environment, it would be awkward in the extreme to maintain shared secrets such as passwords on every client and server that needs to authenticate requests. Furthermore, distributing shared secrets across many systems increases potential vulnerabilities in direct proportion to the number of systems—a problem exacerbated by the “weakest link” phenomenon. Kerberos addresses this requirement by maintaining a centralized database that is distributed across only a few authentication servers. While overall security is critically dependent on protecting this central database, it is much easier to harden a few special-purpose servers against attacks than to protect many general-purpose systems. The central control over authentication secrets also makes it easier to issue new credentials, revoke existing ones, and recover from compromised credentials.
The Kerberos developers assumed that anyone could eavesdrop on network traffic, could claim to be any user, and could set up rogue servers capable of posing as any legitimate service, including the Kerberos services themselves. Encryption was used to prevent eavesdropping attacks, and session keys were introduced along with timestamps to prevent replay attacks. When users (or hosts/services) authenticate to the Kerberos authentication service, the authentication service in turn authenticates itself to the user (or host/service) by proving it knows the previously established shared secret. A by-product of these counter-measures is that Kerberos provides protection against man-in-the-middle attacks, which were generally regarded as infeasible at the time, and for more than a decade after Kerberos was initially deployed. Sadly, man-in-the-middle attacks are no longer mere conjecture, and are all too common in today’s Internet web, which was not designed with a hostile environment in mind.
If you can reasonably join all of the systems to the Kerberos realm, for instance they're all sites within your company, then it should be fine to use. However, it doesn't necessarily encrypt everything; for instance, HTTP would still be unencrypted. Some services such as NFS, though, can encrypt their data flows using Kerberos. Depending on your applications and the data you'll be transmitting, you may still want to use a VPN between sites.
But if your applications are going to be open to the public, then Kerberos probably isn't what you want.