0

We are trying to communicate from an embedded application to a device via HTTPS protocol. We are having difficulty doing so from the embedded application but can do so successfully from say Windows using Curl command. At the firmware level with the libraries we are using (libcurl, openssl) we are seeing an additional TLS header(line) prior to the content of the POST request (but after the HTTPS headers of the POST) whereas that header(line) is not there, when communicating to the device from Windows.

Note we are getting past the TLS handshake sucessfully in both scenarios.

The additional line is the …. 17 03 03 00 4d line displayed in the capture for the HttpsClient below. The first capture shows communication between Windows Curl and the device. The second capture shows communication between embedded application and the device. Locations like https://www.httpbin.org/post have no trouble handling this extra TLS header line as we are able to communicate to that url from the embedded application successfully.

On the embedded application the libraries versions are: Libcurl: 7.41.0 (February 25 2015 release) Openssl: openSSL 1.0.2h

We have not been informed what versions of libraries are being run on the device.

Has anyone dealt with this before or have any ideas on how to get around this?

Here are the captures ... enter image description here

enter image description here

Improve this question

1 Answer 1

Reset to default
4

There is no such thing as a "TLS header line". "17 03 03" in your output just means the beginning of a new TLS frame with application data (type 23, i.e. 0x17, followed by TLS protocol where 0x0303 means TLS 1.2).

The output you show instead means that one client is sending the POST header and body within a single TLS frame while the other client is sending the POST header in one frame and the POST body in a second frame. This is probably caused by the first client using a single SSL_write which includes header and body while the seconds client uses a SSL_write for the header and another for the body. This is both a perfectly valid behavior and you could even split the payload into even more TLS frames. This also means that the server probably needs to do multiple reads to get the full HTTP message (header and body).

If the server has problems with this then the server is broken. Such brokenness is often caused by he server authors trying to write their own HTTP stack by just looking at a few HTTP messages, instead of using an established stack or properly reading the standard. The standard clearly says where a HTTP message starts and where it ends. And since HTTP is set on top of a streaming protocol it might be possible that multiple reads are needed to get the full HTTP message. This is probably the reason why it succeeds with some properly implement server but not with your broken server.

Improve this answer
11
  • "Such brokenness is often" not even an HTTP thing. It's a TCP thing, caused by technically broken human description of TCP being repeated: TCP segmentation is quote often described as a variant of IP fragmentation, at the TCP level (or as AAL5...). Although the explanation will never explicitly say that TCP segment is the analog of IP fragment, that's so strongly implied that pretty everyone much gets that on first reading. And it's absolute pure BS. The very description of TCP as providing sending of messages bigger than MTU, reassembly, etc. is wrong. TCP provides octet stream, period.
    – curiousguy
    Commented Nov 9, 2017 at 6:11
  • The graphics used almost always use the same representation for TCP as for UDP, IP... this is bad bad bad. TCP implements a continuous stream of octets (for for each direction) and read(1) (if you are using the usual Unix API) can return any length of data smaller than what was sent, so there is no guaranteed relation between the number of calls to write(1) on one side and the number of returns from calls to read(1) on the other, unless you send exactly one octet each time.
    – curiousguy
    Commented Nov 9, 2017 at 6:16
  • HTTP/TCP and HTTP/TLS/TCP both exist and HTTP has the exact same operation in both cases. TLS does not just present a stream of octets to the application, but HTTP has no use of anything but a stream of octets. Any additional semantic provided by TLS cannot be used as HTTP is designed for TCP.
    – curiousguy
    Commented Nov 9, 2017 at 6:19
  • @curiousguy: While I agree that a wrong understanding of network protocols is a problem I don't think there is any actual value added to this answer by your extensive (spanning multiple comments) rambling about a wrong understanding of TCP.
    – Steffen Ullrich
    Commented Nov 9, 2017 at 7:14
  • I don't think there is any actual value added to my alleged "rambling" by your short comment. ;)
    – curiousguy
    Commented Nov 9, 2017 at 7:36

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .