If you can put up a simple "type the number in this picture" CAPTCHA and have that stick for 24 hours, you know your enemy is an amateur. You know this sort of primitive device involving bespoke code will slow them down for 24 hours. This could be fun :)
I would make extensive use of stylesheets to hide information in the page code: in two senses, first hide CAPTCHAs, and second hide informational answers. With an aim to sadistically mislead scrapers.
I would write a bit of code on server-side to create fake answers that are believable at first glance, but phoney in ways not easily confirmed. Further, use random seeding or MD5s to make sure the same input always gives the same phoney answer.
Misleading on CAPTCHAs:
Now, the scraper will not realize the first CAPTCHA is suppressed with stylesheets. It will cheerfully solve the CAPTCHA and return the answer with the wrong key. Gotcha. However, just like cracking Enigma, you can't make it obvious that you've busted the code; the scraper must continue to believe it is working.
Misleading on answers:
Present an answer as normal, with a stylesheet around it. The stylesheet hides this result to normal people; the scraper is oblivious that this sheet has the "hidden" property. The answer you present here is the fake. Present the true result afterwards. For bonus points, present the results in a graphic which makes it unscrapeable. Try to conceal this of course.
If you have telemetry (solved wrong CAPTCHA) that this is a scraped query, then don't even bother buying that query result from your service provider. Insert a sleep(t+random) for the typical range of time your service provider takes, then send back a fake answer.
The attacker will believe things are working normally, and only checking for query success, not quality of results. With luck, your attacker won't have logged when each query was made, and is simply dumping the answers into a database. It may take the attacker quite a long time to realize you have poisoned the data, by which time, the entire database would be corrupted, having no idea which entries are valid and which are poison (see the importance of making the fake data look legit?) Even if the attacker timestamped every entry, what a bug hunt! Having to manually check several entries for each day to figure when the data went bad.
And one more thing. Cache true answers, and if a botnet query is in cache, always give the correct answer from cache. So the scraper, troubleshoting, will hit your real website with a browser, and ask for a test number of 213-456-7890. The hiding will work and this will behave like a real query, so you'll compute the real answer and return it. Next, the scraper will tell the botnet to ask for 213-456-7890. To see if the bot gets a different result. You will detect the bot query. If you now give a fake answer, the scraper knows the jig is up, and will iterate on breaking your detection. So since you have the true answer in cache, give it, even in the hidden fields. Now, the scraper is perplexed: the botnet seems to work.
Why and how
Obviously someone finds your data valuable. They would get it from your source, but they don't want to pay for it, so they are scraping you.
- It is possible they are actually a competitor website which does the same thing you do, and they generate a query to you when they get one from their visitor. In essence this is a scheme to use your service but put up their ads. You yourself know the value of that. You can test that by making obscure and different queries on every competitor site and see which queries pop up in your logs.
There are a zillion ways to get CAPTCHAs solved. In the example of a competitor website pulling your data for their customer, they may simply be passing your CAPTCHA onto their customers. There are also ways to trick humans into doing CAPTCHAs for you, such as "solve CAPTCHAs get free porn", or by offering an unrelated service that requires CAPTCHAs for some reason, like an anonymous bulletin board. Everytime someone posts, it sends you a query and gets its poster to solve your CAPTCHA. There is also CAPTCHA solving essentially slavery in the third world.