Firstly, often encryption is terminated at the perimeter by infrastructure which is dedicated to offloading SSL decryption. It makes it much easier to manage when you only have maintain a high degree of key security for a small (proportionally) group of servers which are dedicated to the role. The rest of your regular application servers can operate like normal without worrying about handling these keys.
Secondly, their keys would almost certainly be stored via a Hardware Security Module (HSM). These are dedicated hardware devices with a processor designed for maintaining security and being efficient at performing cryptography.
Finally, Google has its own intermediate CA certificate which it can use to sign its own leaf certificates. This allows them to use certificates with a far shorter expiry than normal, which somewhat reduces the risk of a key being compromised. The actual CA key can be kept locked away in an air gapped bunker and only accessed when it needs to sign a short term leaf certificate.
Remembering also that a CA doesn't need the leaf private key to sign a certificate, they could generate a private key which never leaves a HSM onsite at a remote data centre, then send the public key/CSR to the CA to be signed. They might even generate a unique private key for every single HSM device, that way a key never has to leave the device which generated it.
For example here's a print screen of a current Google certificate:
You'll notice that the certificate is only valid for 3 months, which is significantly shorter than most.
Edit: Google actually describes their policies in detail here.
In the Google PKI, a Subscriber is an individual or an organization capable of using, and authorized to use, the Private Key that corresponds to the Public Key listed in a Certificate
...
The Google Internet Authority servers are located inside of a locked cabinet or cage area in a locked server room. Access to the server room is controlled by badge readers. The private keys for the Google Internet Authority are stored in hardware security modules that are FIPS 140-2 Level 2 that are physically tamper-evident and tamper-resistant.
...
Subscriber Key Pairs are generated (i) by the Subscriber by software supplied by their device/operating system, or (ii) by an authorized member of Google’s Information Security Team.
...
Subscribers provide their public key to Google for certification through a PKCS#10 Certificate Signing Request. The preferred transfer method for sending this information is HTTP over Secure Sockets Layer (SSL).
Basically that's exactly how a regular CA works, except it's all conducted internally within Google.
