If the scanner tray is considered as an interface, and it accepts input (basically it is its main functionality), could it be hacked using malicious code written on a piece of paper?
The answer entirely depends on implementation of scanning process in the printer. Modern printers are in essence computers and they are much more powerful than their predecessors from earlier days.
So, the question boils down to "is it possible to hack a computer by using an image"?
The answer is yes, because creating exploit-free software is almost impossible as described in the answers here: Is exploit-free software possible?
Image handling libraries happen to have vulnerabilities. An attacker could entice a user to open specifically crafted image which would exploit a vulnerability on victim's computer, and therefore affect it in some way.
So, if printer's scanning process involves some sort of processing of scanned images, and its software contains bugs, then we can assume this vulnerability can be exploited by a knowledgeable attacker.
could it be hacked using malicious code written on a piece of paper?
The printer won't execute code written on a piece of paper. However, there exists probability that printer's software used for processing of scanned images contains bugs that make printer misbehave if it encounters a certain image.
The attack surface depends on how much processing the printer does with the documents that are being scanned.
The result of this is hard to tell. It depends a lot on the printer and its software, its capabilities.
VL-80's answer is good, but there are conditions under which the answer would be a clear no for "is it possible to hack a computer by using an image?" - for example, if all the code processing the image has no branches or array offsets dependent on values from the content of the image. If all the printer did was copy the image or store it in uncompressed form, that might be the case.
However, there are at least 3 places in which there are potentially exploitable code paths dependent on image content:
- Compressing the image to save it, or as an unnecessary transport step internally.
- Making decisions on dither patterns to reproduce the content in the most visually accurate way.
- Backdoor code in the printer placed there by the manufacturer to assess whether the image you're scanning is money, and if so, alter the result or refuse to scan it so as to prevent use in counterfeiting.
This last one is almost surely full of complex, low quality, code that's poorly understood by the people working on it, and is where I would expect to find exploitable vulnerabilities. Exploiting them would involve partially reproducing the visual signatures in currency, but in ways the algorithms don't expect to see.
I will be the contrarian here.
could it be hacked using malicious code written on a piece of paper?
While bugs in image processing libraries do happen, writing code on a piece of paper, scanning it, and expecting it to execute on the scanner for hacking purposes is the kind of thing that only happens in (bad) movies. Especially when it comes to making the scanner literally execute code that is written on a piece of paper, we're far past the realm of probability for accidental bugs and entering a realm where this would have to be programmed intentionally - and even then it wouldn't be very useful.
Of course we can loosen that requirement and ask about more "standard" weaknesses instead of literally executing code on a page. From that perspective there are three main areas of concern:
Vulnerabilities in imaging hardware - highly unlikely
You'll have the most success if you can do something to "trick" the imaging hardware into behaving in unexpected ways. If you have full control over the hardware of course this is entirely plausible - shoot lasers at it, replace chips, etc... Except that's obviously not what we're talking about. We're talking about taking over the system by scanning a "malicious" page using the system's normal operations.
Realistically though, there is no reason to think you'll find any help in hardware issues. I'm speaking here from my own years of experience performing extensive image processing on raw images straight out of hardware (in my case I was performing image processing on data coming out of CCDs, so a slightly different use case). In my experience with CCDs, the CCDs would always generate valid images (valid in this case means only that it produces images in the proper format - not that the image itself is necessarily meaningful). It was quite impossible for them to do anything else due to physical limitations of the hardware, even when exposed to sources they weren't intended to be used with (which isn't an option here anyway).
Note that I'm not trying to say that the hardware might not have issues or "hiccups". However, there is no reason to think that a specially crafted piece of paper might cause trouble for the actual imaging hardware - generally they just don't work like that.
Vulnerabilities in image processing libraries - also a no-go
This has been mentioned in previous answers. There have been a number of instances of security vulnerabilities in image processing libraries, sometimes even to the extent of a code-execution vulnerability.
However, that doesn't mean that this is a serious concern for scanners - in fact, it isn't. The trouble is that, for vulnerabilities in image processing libraries to be exploitable, the attacker has to have substantial control over the input into the image processing library. That simply isn't the case here. Instead the attacker has control over the input to the scanning hardware. The image processing library then works off of what it is given by the hardware. Therefore, to exploit a bug in the image processing library would require not just a bug in the image processing library but also a flaw in the imaging hardware that allows the attacker's "payload" to somehow survive the imaging process (which is itself subject to many sources of random error and bias, likely rendering a consistent exploit completely impossible).
Can I "prove" that such a thing is impossible? Of course not. Crazy things do happen after all, and many real world exploits happen as a result of failures at multiple levels. However, just because something might be considered "plausible" doesn't mean that it should be a real security concern, or even that it has ever happened. I certainly wouldn't waste my time trying to check the security of my office scanners/copiers for these kinds of vulnerabilities.
OCR capabilities - also not a concern
The possibility of weaknesses in OCR have been brought up. This is also not a serious concern. There are two important realities that make this a non-starter:
- Most copiers don't come with built in OCR software
- RCE vulnerabilities in OCR software are virtually unheard of.
Honestly, I'm just going to leave it at that.
Sure, vulnerabilities have been found in very unexpected places, and you never know what crazy things will end up in code/hardware. However, this is security.stackexchange, not worldbuilding.stackexchange, so I think an answer grounded in reality is important. Therefore I want to emphasize that, while just about anything is possible when it comes to security weaknesses, this is an attempt to hack a system at what is effectively it strongest point. The imaging hardware is effectively a powerful input normalization system, and getting a malicious payload through it should be nearly impossible. Moreover, the kinds of weaknesses that are typically present in image processing libraries rely on making modifications to images in ways that someone "attacking" through the imaging hardware physically cannot perform.
Possible? Only in the sense that any event with a non-zero probability is possible. A legitimate concern? Definitely not. I wouldn't even describe it as "theoretically" possible.
In 2017 researchers at the Cyber Security Research Center at Ben-Gurion University in Israel demonstrated a scanner attack using lasers. Drone-mounted lasers controlled by an attacker were used to communicate with malware installed behind an air-gapped system, e.g. through a window into an office.
Article here: https://arxiv.org/abs/1703.07751
This isn't exactly what is being asked about and it doesn't involve paper codes, but it may be relevant concerning scanner vulnerabilities.
Related pop-sci article here: https://www.digitaltrends.com/cool-tech/hacked-scanner-laser-drone-method/
Theoretically possible? Yes, sure
With the scanner running the human-readable text written there? Not really, as mentioned in other answers. You would need to have a printer programmed to do something at least slightly similar. For instance, if this scanner had an Alexa assistant that expected written queries.
However, I don't think "malicious code written on a piece of paper" needs to be taken that way. The code may take many forms. In fact, the main problema for real-world explotaition will be, once you have X scanned data launching your exploit, to get a piece of paper that gets scanned to that code, since a difference of one bit will result in a completely different shellcode (a non-working one, actually). E en with a monochrome low resolution scan, you will need high prevision crafting the piece of paper.
As for which vulnerable code that could to the exploit, and this enters more into the worldbuilding realm, I would suggest the feature to prevent money photocopies.
Many photocopies detect a pattern on bank notes (the "EURion constellation) in order to refuse copying money.
This means that there is a secret specification of how the circles may be located (although a large part has been reversed), a corresponding secret algorithm used by the scanner manufacturer to detect them (whose publication could show how to 'game' the detection) and in general a quite secretive code (probably only a few people will have access to it) which won't be highly tested (in particulsr, due to the above reasons, the tests will likely consist in scanning a bank note or certain test paper).
Also note that this code will be running after the hardware copied the image. So you could imagine an exploit pattern that was located in such a way (e.g. at the begin/end of the scan data), with a naive EURion code that on finding a circle candidate, looks for another one at X location foñrom it -with no bounds check-. If that vulnerability leaded to code execution, you would only need to jump anywhere into the image data (in memory at this point), where there would be a large nop-sled leading to the shellcode (that, if it is a printer could then execute a later postcript stage).