Is there a way to use sqlmap over telnet or do I have to manually create a web wrapper that will send the payload over a parameter and output the response?
SQLmap has no native support for Telnet, and it's not clear that it would handle typical telnet (or similar protocol) responses in a useful way: it makes use of (among other things) the status code for responses, path components of the URL and encoding of HTTP parameters in request bodies. None of these would apply to a telnet connection.
You can use SQLmap to directly scan a database management system, using the
-d switch, but this uses whatever protocol the DBMS uses, and tends to require credentials to access the DBMS.