OBJECTIVE: We have a requirement to run active scan for detecting and monitor phishing websites for a particular brand.

We want to be as real-time as possible. That would mean that using sites like Phishtank would not suffice.


  1. We are thinking to do this based on some parameters including, but not limited to images, content, security certificates, etc. Are there any existing tools which can be worked upon or modified to run active crawling and enlisting the websites that match our phish pattern?

  2. Not sure if quite feasible or even possible - is to monitor public DNS Servers for new domain registrations (is this possible? How?) and crawl these websites, run our phishing detection algorithms.

UPDATE: I am even open to ideas to achieve the above objective.

  • Web crawling is unlikely to be a viable strategy for finding phishing websites, because they are usually not linked from other websites, only from the emails they send. Maybe it would be better to set up a few email accounts as honeypots and try to get them listed by as many spammers as possible so they receive lots of phishing mails?
    – Philipp
    Jun 17, 2014 at 7:57
  • Yes, crawling would be a tedious task, I agree. That's one idea though. I am open to other ideas. I am updating my question. Jun 17, 2014 at 9:38
  • Why do you need it in real-time? You think you can have coverage of the entire Internet looking for particular files?
    – schroeder
    Jun 17, 2014 at 14:30
  • 1
    Are you aware that newer phishing techniques use websites that are only up for a matter of hours? Is your requirement to catch even these?
    – schroeder
    Jun 17, 2014 at 14:32
  • I understand its not possible to cover entire Internet, and that is why I put it as 'near Realtime'. :) Yes my requirement would be to try and catch even those up for only a matter of an hour. Jun 18, 2014 at 5:30

1 Answer 1


Note: I am assuming you are working for/with that brand.

Most phishers perform the minimum work they can to create the fake page, which means they leave links pointing to your javascript, css, images…

By watching the server logs and the referrer provided by your visitors, you will be able to detect new phishing sites using that code in almost real time.

Also, you can intentionally put absolute urls in the html, css and javascript to increase the odds that it leaves some references to your servers.

  • That's a very good idea, and I will use this. Additionally, we would still like to cover those websites who would simply use brand logos with their own HTML forms. Of course catching these even in near real-time is difficult as they might use saved logos. Any ideas on this? Jun 18, 2014 at 5:35
  • You could also check from which websites people is landed onto your page (as they typically -but not always- redirect to the legit page as final step), although (a) there will be many good sites linking to your page and (b) they can code the page to avoid sending the Referer header.
    – Ángel
    Jun 25, 2014 at 9:42

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .