I have heard that the reason the image portion of ReCaptcha contains so many driving questions is that Google is using it to train AI for their self-driving cars. But to make it an effective security measure, wouldn't Google need to verify that the squares the user clicked on contained stop signs or street lights or whatever they asked for? That would mean they already know what is in each image, so how do they gain new information from it? How can they verify the information is correct without knowing beforehand?

  • I think it's the same question security.stackexchange.com/questions/78807/…
    – Danielr
    Commented May 27, 2019 at 18:16
  • 2
    Possible duplicate of How does Google's "No Captcha reCaptcha" work?
    – ThoriumBR
    Commented May 27, 2019 at 18:37
  • 2
    I don't think this is a duplicate at all. Those questions are asking about the imageless "check this box if you're not a robot" things Google has started doing. This question is about the "click all squares with a street sign" image-based challenges, specifically how would this method accomplish all three of "fool bots", "train Google's driving AI", and also "be verifiable by Google" at the same time.
    – Ben
    Commented May 27, 2019 at 19:33

2 Answers 2


Very simple: They ask multiple people.

When you need to fill out recaptcha, they show you images they have shown many people before, and for those they know the correct answer and they score you on those. But they also show you images which they have not shown many people before. They do not have enough answers for those images to know what the correct answer is. They do not score your answer for those images, but they record your answer. When enough answers are collected for an image, Google become reasonably sure about the correct answer and can start using this image for scoring.

Obviously, it's more complicated than that (for example, they use their machine learning on the images, which is probably at least 90% accurate, so they have a good idea about the correct answer even if you are the first person shown this image on recaptcha), but that's the idea.


Google will show for example six pictures which do or do not contain a stop sign.
For five of them they indeed already know the answer (verification pictures), for one they are not sure themselves (AI training picture). They will not tell you which one is the training picture.

The trick is they will require you to identify the 5 verification pictures correctly but they will accept any answer for the training picture. Your answer to the training picture is only used as input towards the AI training.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .