Information Security Stack Exchange is a question and answer site for information security professionals. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Could entering sensitive keyboard input in public places with camera systems equipped with a robust AI possibly leak keystrokes via eye-tracking technology -- assuming the typist is using the hunt/peck method? This would assume the system uses some kind of spacial guess-work, creating a virtual map of the keyboard by using conjectural spacial probability, perhaps based occurrence-frequency of right vs left vowels, etc.

This of course also assumes the keyboard is obscured either by angle or the laptop lid.

share|improve this question

closed as primarily opinion-based by tylerl, NULLZ, Scott Pack, TildalWave, Lucas Kauffman Jul 23 '13 at 6:48

Many good questions generate some degree of opinion based on expert experience, but answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise.If this question can be reworded to fit the rules in the help center, please edit the question.

No need for AI for what you describe, and remember, watching people's arms can also be useful, assuming they are bare. – Rory Alsop Jul 22 '13 at 14:08
That's possible a long time ago. Check this innocent demo in an Augmented Reality app. – HamZa Jul 22 '13 at 14:11
Any links to proposed methods of mitigating such threats, aside from the obvious sports coat and sunglasses? – Elmer Jul 22 '13 at 14:18
Perhaps only biometrics would do, but could a gesture-activated protected password be a start? – Elmer Jul 22 '13 at 14:28
up vote 5 down vote accepted

When you start to use the word "assume" three times or more, you might be too broad and might get hit by a lot of individual opinions. Anyway…

Does the technique your assumptions describe actually exist?

Yes, it exists. It's part of what's commonly known as "body tracking surveillance techniques".

Will you ever meet it in real life?

Probably not — unless governmental institutions like the NSA learn to love you.

Is eye-tracking the only problem in your scenario?


Based on the assumptions of your scenario descriptions, you should also look into "body tracking" and "statistical identification of movement patterns". To give you a simple hint on these kind of things: "shoulder movement" on it's own (no matter how minimal it is) can already provide enough identification vectors to reproduce the keys you have typed with a high probability of success.

Yet, physical tracking methods are rather impractical on a "one-person" scale as there is too much personel (specialists evaluating the data) and hardware (needed surveillance equipment) involved.

Why would someone want to use complicated techniquest when there are much simpler ways to gain access to your passwords. Let's - just for fun - assume someone really wants to get your password/pin/code/whatever using physical tracking methods… that someone will have to gather additional reference data and - even more important - be sure that the resources invested are actually worth the result.

So, unless you're working for "el presidente" and got some really cool access level to classified information - most adversaries will most probably fall back on easier methods to gain access to your passwords. After all, the goal of "them" will always be to invest a minimum of resources to reach access to your passwords. Trust me, no one will ever hire two to three infosec specialists with hi-tech equipment just to steal your top-secret Gmail password.

But let's suppose your password is really that important. Let me list a few security problems you'll have to take care of first:

  • Is that waitress really cleaning that table (or is she watching what you type)?
  • Did you check the tables that surround you (and the people sitting near you who could be watching you)?
  • Are you sure there are no hidden cameras recording you from intrusive angles?
  • Are you sure there is no hidden microphone underneath your table to record your keystrokes (which can aid reconstructing the letters you type using sound-to-microphone distance vectors)?

and last but not least

  • Are you really sure you want to use such an important password in a public place like this?

Wrapping it up...

Let's be honest — no one will use eye-tracking when there are plenty of alternatives to get what you think is worth securing. Compared to eye-tracking worries you describe, those alternatives will certainly provide easier options to gain access to your secured data.

But assuming the data you plan to enter in your device is really that confidential, I would take a coffee-to-go and look for a more trusted place to enter that data/password/pin/code/whatever.

In all other cases, it's time for a little reality check to prevent the security-related paranoia that hits us all every now and then…



share|improve this answer
You guys are really great. Thanks to all for taking the time lead me in the right direction. Excellent feedback. – Elmer Jul 22 '13 at 16:03
I see no option to accept a particular reply, but yours suffices just fine. – Elmer Jul 22 '13 at 16:09
@Elmer To accept an answer, you simply have to click the "checkmark" on the left side of the answer (beneath the up-down rating arrows). It'll turn green as soon as you've clicked it. Here's an example: – e-sushi Jul 22 '13 at 16:11
@Elmer There might possibly be a delay in your ability to accept answers because you are a new user; if so, I would expect to see the option very soon (in the form of a gray clickable checkmark). See also How does accepting an answer work? – apsillers Jul 22 '13 at 16:14
@apsillers Thanks for sharing that link. Gotta bookmark that one to have it handy next time I want to help new members. – e-sushi Jul 22 '13 at 16:16

Eye-tracking, in this context, makes sense for the attacker only if he could plant a camera which has a very good view on the user's eyes, but not on the user's fingers. That's a rather restrictive scenario. ATM skimmers usually achieve the reverse, i.e. a camera which sees the fingers and not the eyes. Indeed, there is an advantage for an attacker's camera to not see the victim's eyes: when a camera sees the user's eyes, the user can see the camera...

Even if the camera could see the user's eyes with a good enough resolution to track the eye movements, then chances are that the same camera could simply see the reflected image of the keypad in the user's glasses or even in the user's eyeball (using the glasses is much easier, so that's a bad point for sunglasses as a countermeasure, by the way).

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.