I'm working quite some time now with TLS 1.3 implementations (OpenSSL, WolfSSL), but I can't find anywhere in the TLS 1.3 drafts if alert messages should be encrypted or not. I personally thought only alerts send after the Client- and ServerHello will be encrypted, but I could not find any proof for this.
The situation in TLS 1.3 regarding encryption of alerts is no different then with TLS 1.2. The current draft 26 includes the same statement as you can find in the RFC of TLS 1.2:
Like other messages, alert messages are encrypted as specified by the current connection state.
In Appendix A you also find the state diagrams which have additional information:
... clients may send alerts that derive from post-ServerHello messages in the clear or with the early data keys. If clients need to send such alerts, they SHOULD first rekey to the handshake keys if possible.