How does OS detection work in practice?

Question

I was wondering how OS detection really works on Nmap. I know it is about to send different tcp probes to see how the target machine responds to them.

But what kind of probes would distinguish a Linux system or a Windows system? A Linux 2.4 or 2.6 system? Windows XP or Windows 7 system etc?

Answer 1

you can check the following links NMap OS Detection and Paper About Nmap Detection you will find on them different methods used for OS detection By Fyodor the Creator of Nmap and if you want to dig dive you can purchase his book from amazon.

From NMap OS Detection:

...dozens of tests such as TCP ISN sampling, TCP options support and ordering, IP ID sampling, and the initial window size check, Nmap compares the results to its nmap-os-db database of more than 2,600 known OS fingerprints and prints out the OS details if there is a match