A local copy of the certificate really makes no difference one way or another - anybody who knows your domain can pull a functionally identical copy.
The private key is needed by your server, notably, to negotiate the key-agreement protocol
When Alice and Bob have a public-key infrastructure, they may digitally sign an agreed Diffie-Hellman agreed key, or exchanged Diffie-Hellman public keys. Such signed keys, sometimes signed by a certificate authority, are one of the primary mechanisms used for secure web traffic (including HTTPS, SSL or Transport Layer Security protocols)
It also ensures the confidentiality of the messages it encrypts more generally, using Public Key Encryption:
The encryption process of using the receivers public key is useful for preserving the confidentiality of the message as only the receiver has the matching private key to decrypt the message. Therefore, the sender of the message cannot decrypt the message once it has been encrypted using the receivers public key. However, PKE does not address the problem of non-repudiation, as the message could have been sent by anyone that has access to the receivers public key.
You could arguably delete both once they are loaded into memory, if they are assured of remaining there - i.e. your server will never suffer any unexpected event.
That, however, seems a bad bet to make.
As mentioned by @Arminius, the key can also be used for things other than TLS, such as PKI-related activities.
