How do we come with True Positives and False Negatives rates when creating signatures in IDS? How do we measure the signatures efficiency? I've seen so many papers that discuss the same, but how do they come up with these numbers?
How do we measure the signatures efficiency?
You don't want too much false positives, or your users will be impacted for being denied access to legitimate sites. On the other hand, false negatives allows attackers to bypass the filters and attack your infrastructure. So an IDS with the lowest false positives/negatives rates is better than the others.
how do they come up with these numbers?
They create a list of URLs, both malicious and clean, and send to the IDS, recording the result. This is how they get the amount of false positives and negatives.
You can create the same list by checking your log files and manually labelling them as malicious or not. Check those labels against the IDS log file, and you will end up with the Confusion Matrix.