When executing the command
nmap -sn -Pn --script http-robots.txt nmap.org, I get no results back from the specified NSE script "http-robots.txt.nse"!
The script should show all the "disallow" entries in the "robots.txt" file on a given target domain, but nothing shows whatsoever no matter what domain I specify! Here's the output of the command:
Starting Nmap 7.31 ( https://nmap.org ) at 2016-11-06 18:01 EET Nmap scan report for nmap.org (188.8.131.52) Host is up. Other addresses for nmap.org (not scanned): 2600:3c01::f03c:91ff:fe98:ff4e rDNS record for 184.108.40.206: ack.nmap.org Nmap done: 1 IP address (1 host up) scanned in 1.52 seconds
Any thoughts on what the problem might be?