I have searched online about this topic and the only information I got is that TCP SYN/ACK can be used to know the type of the target OS. If that is true, I don't understand how this occurs? Does TCP handshaking lead to OS fingerprinting?
OS detection goes a lot further, using all kind of data of the IP and TCP headers fields. If you're into reading, "Silence on the Wire" by Michal Zalewski addresses a lot on this topic.