You didn't mention what operating systems are involved (I presume Linux on the rPi) nor whether you have access / are comfortable with modifying the code.
While you could apply encryption in the client and server it would be a lot simpler to just implement a tunnel between the 2 devices - you point your current client at a new service on the client host and restrict the server to listening to a complementary service running on the rPi. Between the 2 new services the connection is encrypted.
There are lots of choices for the tech you use the new services, but the most practical choices are based on either TLS or ssh. For TLS you can use stunnel or stud (the same service, configured differently at each end) for ssh you use port forwarding from an ssh client such as putty/plink/ssh to the opensshd on the rPi. In both cases you should configure the firewall on the rPi to only accept connections on port 19000 from localhost.
Both methods rely on key pairs but the TLS based mechanism has a lot more choices and potential complexity. OTOH, if you are using a MSWindows client, the stunnel user interface is perhaps easier to live with than putty or plink.