Can http referrer value be used to stop spam bots?

Question

I have a public sign up site, and when they click submit, a verification text message goes out to the phone number specified in sign up by calling an aspx page via ajax. I don't want spam bots sending out text messages that I pay for.

I noticed that google's noCaptcha feature with the simple check box has a lot of great things. But the most important feature it has is that it only accepts requests from ip addresses or domain name you designate. Good. And to do that, it has to check the http referer in the request. So what if the browser sending it or the firewall surrounding it erases http referer? Then I guess captcha doesn't work in that situation and gives back improper ip address or domain name error, or whatever it gives out and I have to handle that.

So my question is why does noCaptcha even need to be used? If I can check the referer ip address or domain name in the page that receives the request to send out the text message, and reject (or do whatever) if referrer is blank or not my ip address or domain name (because a spam script will be executing on the spammer's server or ip address or domain name), then am I not doing the same thing as noCaptcha and don't need it?

Seems like if a spammer's script is running on another ip address or domain name invoking the js ajax calls it found on my page via scraping, the referrer ip address or domain name will either be blank, or the real spammer's address, as it seems that http referrer can't be spoofed, only excluded or be changed through proxies, none of which will be my ip address or domain.

Either way, is this not just as effective and gets to the bottom line? If noCaptcha can't accept requests without a proper and correct http referer, then is my check no less than theirs? Either solution will not work if there is a firewall that prevents http referrer from being specified or a spam bot script that does not specify it. So what am i buying (even though it's free) from noCaptcha if it has the same limitations of the idea I am proposing where the check is done on the requested page that will send out the text message?

I'm looking for people to tell me why my captcha replacement approach will not actually work

Answer 1

First off, your understanding of the referrer field is wrong. Google uses that field to ensure that another site isn't using your API key. If the referrer field gets stripped, the user won't be able to complete the captcha, which in turn means they can't submit the form, which means no text message sent.

Anything and everything can be spoofed, just not in the browser. If I were to attack your site, I wouldn't write an AJAX script and run it in the developer tools. I'd use a platform like NodeJS to write the bot, giving me fine tuned access over the request I make. Heck, I can even telnet to your server and manually type in a fake referrer. A malicious user could figure out that all you're looking for is a specific http referrer entry, then write a script to spam your signup page and drain your text money. They could even be in cahoots with a premium text provider, and get money out of you.

Googles noCaptcha is a lot more complicated than you think. The reason why you only check the box and done is because Google has already determined you are a real person. They use very advanced heuristics for this, and take into account a lot of variables, more than what your implementation does. Their risk engine tracks stuff like how you move the mouse towards it, how many other captchas did you complete, are you constantly looking at a single captcha field. If you were to switch into incognito mode or private browsing, you'd most likely be shown a challenge before you can check the box.

In short, you're saying "i don't need captcha, i fully trust that the person talking to me isn't lying about where they're coming from"

Answer 2

Can http referrer value be used to stop spam bots?

No, because the referrer is just a header and is easily forged by a bot.

I noticed that google's noCaptcha feature with the simple check box has a lot of great things.

Actually it isn't a checkbox, it's a DIV with some overlays.

But the most important feature it has is that it only accepts requests from ip addresses or domain name you designate.

Not sure how you "noticed" this. Did you find that in a document? Because it's nothing remarkable; you can easily set up an IP white list of your own using only IIS-- no noCaptcha required.

why does noCaptcha even need to be used?

It's a sophisticated risk engine with conditional escalation steps. Details. I doubt you could effectively recreate this yourself.