Could entering sensitive keyboard input in public places with camera systems equipped with a robust AI possibly leak keystrokes via eye-tracking technology -- assuming the typist is using the hunt/peck method? This would assume the system uses some kind of spacial guess-work, creating a virtual map of the keyboard by using conjectural spacial probability, perhaps based occurrence-frequency of right vs left vowels, etc.

This of course also assumes the keyboard is obscured either by angle or the laptop lid.

  • No need for AI for what you describe, and remember, watching people's arms can also be useful, assuming they are bare.
    – Rory Alsop
    Jul 22, 2013 at 14:08
  • That's possible a long time ago. Check this innocent demo in an Augmented Reality app.
    – HamZa
    Jul 22, 2013 at 14:11
  • Any links to proposed methods of mitigating such threats, aside from the obvious sports coat and sunglasses?
    – Elmer
    Jul 22, 2013 at 14:18
  • Perhaps only biometrics would do, but could a gesture-activated protected password be a start?
    – Elmer
    Jul 22, 2013 at 14:28

2 Answers 2


When you start to use the word "assume" three times or more, you might be too broad and might get hit by a lot of individual opinions. Anyway…

Does the technique your assumptions describe actually exist?

Yes, it exists. It's part of what's commonly known as "body tracking surveillance techniques".

Will you ever meet it in real life?

Probably not — unless governmental institutions like the NSA learn to love you.

Is eye-tracking the only problem in your scenario?


Based on the assumptions of your scenario descriptions, you should also look into "body tracking" and "statistical identification of movement patterns". To give you a simple hint on these kind of things: "shoulder movement" on it's own (no matter how minimal it is) can already provide enough identification vectors to reproduce the keys you have typed with a high probability of success.

Yet, physical tracking methods are rather impractical on a "one-person" scale as there is too much personel (specialists evaluating the data) and hardware (needed surveillance equipment) involved.

Why would someone want to use complicated techniquest when there are much simpler ways to gain access to your passwords. Let's - just for fun - assume someone really wants to get your password/pin/code/whatever using physical tracking methods… that someone will have to gather additional reference data and - even more important - be sure that the resources invested are actually worth the result.

So, unless you're working for "el presidente" and got some really cool access level to classified information - most adversaries will most probably fall back on easier methods to gain access to your passwords. After all, the goal of "them" will always be to invest a minimum of resources to reach access to your passwords. Trust me, no one will ever hire two to three infosec specialists with hi-tech equipment just to steal your top-secret Gmail password.

But let's suppose your password is really that important. Let me list a few security problems you'll have to take care of first:

  • Is that waitress really cleaning that table (or is she watching what you type)?
  • Did you check the tables that surround you (and the people sitting near you who could be watching you)?
  • Are you sure there are no hidden cameras recording you from intrusive angles?
  • Are you sure there is no hidden microphone underneath your table to record your keystrokes (which can aid reconstructing the letters you type using sound-to-microphone distance vectors)?

and last but not least

  • Are you really sure you want to use such an important password in a public place like this?

Wrapping it up...

Let's be honest — no one will use eye-tracking when there are plenty of alternatives to get what you think is worth securing. Compared to eye-tracking worries you describe, those alternatives will certainly provide easier options to gain access to your secured data.

But assuming the data you plan to enter in your device is really that confidential, I would take a coffee-to-go and look for a more trusted place to enter that data/password/pin/code/whatever.

In all other cases, it's time for a little reality check to prevent the security-related paranoia that hits us all every now and then…


via http://xkcd.com/538/

  • You guys are really great. Thanks to all for taking the time lead me in the right direction. Excellent feedback.
    – Elmer
    Jul 22, 2013 at 16:03
  • I see no option to accept a particular reply, but yours suffices just fine.
    – Elmer
    Jul 22, 2013 at 16:09
  • @Elmer There might possibly be a delay in your ability to accept answers because you are a new user; if so, I would expect to see the option very soon (in the form of a gray clickable checkmark). See also How does accepting an answer work?
    – apsillers
    Jul 22, 2013 at 16:14
  • @apsillers Thanks for sharing that link. Gotta bookmark that one to have it handy next time I want to help new members.
    – e-sushi
    Jul 22, 2013 at 16:16

Eye-tracking, in this context, makes sense for the attacker only if he could plant a camera which has a very good view on the user's eyes, but not on the user's fingers. That's a rather restrictive scenario. ATM skimmers usually achieve the reverse, i.e. a camera which sees the fingers and not the eyes. Indeed, there is an advantage for an attacker's camera to not see the victim's eyes: when a camera sees the user's eyes, the user can see the camera...

Even if the camera could see the user's eyes with a good enough resolution to track the eye movements, then chances are that the same camera could simply see the reflected image of the keypad in the user's glasses or even in the user's eyeball (using the glasses is much easier, so that's a bad point for sunglasses as a countermeasure, by the way).

Not the answer you're looking for? Browse other questions tagged or ask your own question.