monlist is a debugging command that allows to retrieve information from the monitoring facility about traffic associated with the NTP service.
The reference implementation of NTP [...] allows users to request a list of hosts with which the NTP daemon ntpd communicated recently. The list, called "monlist" has a size limit of 600 entries and contains the IP addresses of the last NTP clients or servers the instance has talked to.
You don't find it explicitly mentioned in any RFC as it is a NTP mode 7 request. RFC 1305 explains that implementations may use "reserved mode 7 for special purposes, such as remote control and monitoring".
ntp-monlist NSE script also has some information:
Monitor data is a list of the most recently used (MRU) having NTP associations with the target. Each record contains information about the most recent NTP packet sent by a host to the target including the source and destination addresses and the NTP version and mode of the packet. With this information it is possible to classify associated hosts as Servers, Peers, and Clients.
Besides the information disclosure problem,
monlist has a history of vulnerabilities such as CVE-2013-5211 which allowed DoS attacks by traffic amplifiaction via
monlist queries with a forged source address.