Network scanning tools such as nmap have the capability to infer operating system version based upon lower-level system details. EG: TCP/IP stack behavior, etc. Are there any known methods to trick these tools to either report the wrong OS or to confuse them so they cannot correctly detect an OS?
There are a wide range of ways to trick OS identification functions, and they work to varying degrees, depending on how much functionality is built into the tool.
For example, nmap doesn't just use one identification mechanism, but a few which feed in to it's confidence %age announced in it's OS identification.
Nmap sends a series of TCP and UDP packets to the remote host and examines practically every bit in the responses. After performing dozens of tests such as TCP ISN sampling, TCP options support and ordering, IP ID sampling, and the initial window size check, Nmap compares the results to its nmap-os-db database of more than 2,600 known OS fingerprints and prints out the OS details if there is a match.
Some tools look for only one signature, for example banner grabbing, so will be easier to fool - by simply spoofing the banner or signature.
So if you want to fool OS detection, you will want to look at a pretty comprehensive set of TCP frame sizes, keepalive functionality, packet number sequences, service banners etc etc
And change them all... this can be pretty difficult.
Instead, do you want to think about using some kind of proxy - depending on how it is configured, the scanning tool is likely to identify the OS on the proxy, which may be enough to reduce or remove the risk you are trying to mitigate.
- Firewall IP addresses that should not be accessing you.
- Turn off ICMP echo.
If you want something more sophisticated, you can use a product such as IP Personality on linux machines to change essential TCP/IP responses.
Keep in mind that such products change your machines TCP/IP behavior and might cause interoperability problems with other machines that expect a specific behavior. Also, this does not help with passive fingerprinting, which looks at the packets you send (passively) to guess your OS (e.g. from banners sent, services requested, etc.)