2

When executing the command nmap -sn -Pn --script http-robots.txt nmap.org, I get no results back from the specified NSE script "http-robots.txt.nse"!

The script should show all the "disallow" entries in the "robots.txt" file on a given target domain, but nothing shows whatsoever no matter what domain I specify! Here's the output of the command:

Starting Nmap 7.31 ( https://nmap.org ) at 2016-11-06 18:01 EET
Nmap scan report for nmap.org (45.33.49.119)
Host is up.
Other addresses for nmap.org (not scanned): 2600:3c01::f03c:91ff:fe98:ff4e
rDNS record for 45.33.49.119: ack.nmap.org

Nmap done: 1 IP address (1 host up) scanned in 1.52 seconds

Any thoughts on what the problem might be?

2

Drop the -sn flag to let nmap do a port scan or specify a port.

The robots.txt is specific to a HTTP service running on a given port. Without checking ports, nmap is unable to identify HTTP services where it could apply the http-robots script.

From the docs:

With the -sn option it is possible to run a script scan without a port scan, only host discovery. In this case only host scripts will be eligible to run.

Drop the flag and you're fine:

$ nmap -Pn --script http-robots.txt.nse nmap.org                                                                                                                                                       

Starting Nmap 7.31 ( https://nmap.org ) at 2016-11-06 17:13 CET
Nmap scan report for nmap.org (45.33.49.119)
Host is up (0.21s latency).
Other addresses for nmap.org (not scanned): 2600:3c01::f03c:91ff:fe98:ff4e
rDNS record for 45.33.49.119: ack.nmap.org
Not shown: 991 filtered ports
PORT      STATE  SERVICE
22/tcp    open   ssh
25/tcp    open   smtp
70/tcp    closed gopher
80/tcp    open   http
113/tcp   closed ident
443/tcp   open   https
| http-robots.txt: 4 disallowed entries 
| /favicon/tiles/ /favicon/dim/ /favicon/dim-nogzip/ 
|_/mailman/
554/tcp   open   rtsp
7070/tcp  open   realserver
31337/tcp closed Elite

If you know which ports you're looking for you can skip the enumeration:

$ nmap -p 80,443 --script http-robots.txt.nse stackexchange.com    

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.