I have a certificate issued from Let's Encrypt.
Can I create a key and certificate for my own purpose (i.e. an OpenVPN server, or web server with internal domain name/IP address) and sign it with the Let's Encrypt certificate?
Usually no, only certificates marked as being a CA can issue certificates. (or, more accurately, you can do that, but no vpn client or web browser will trust it.)
To see if your certificate is a CA, open it and look at the Basic Constraints field; a CA will look like this
while an End Entity will look like this:
End Entities are not allowed to issue certs, and good luck getting Let's Encrypt to give you a CA cert...