On a general basis, you should not forward X11 display from untrusted environments to your local environment. X11 is an old technology which was not conceived with security in mind from the beginning, on the contrary it was designed to allow every application to communicate with each other.
Was does "communicate" mean? As you said, it could grasp you keystrokes. It could also take screenshots. It could interact with other windows, inject keystrokes, mouse moves, etc., etc., etc.. Well, I guess you got it: allowing an untrusted system to export X11 display to your local server is a bit like passing your screen, keyboard and mouse to this untrusted system. You are right in the sense it may not be able to access your local files directly, but it will be nearly equivalent.
Some work has been done in order to mitigate the risk though. X11 indeed include a distinction between a trusted and untrusted clients. An untrusted client should be confined in a subset of actions designed to prevent most of the nefarious actions described above.
SSH integrates this mode, even if some Linux distribution disable it and consider all X11 client as trusted for the end-user convenience (!). You may want to check the
ForwardX11Trusted parameters in your SSH client configuration files:
ForwardX11 tells if X11 is enabled by default when you establish an SSH connection (you must use the
-x (lower-case 'x') command-line parameter to disable it on-demand),
ForwardX11Trusted tells if, when you enable X11 forwarding, the client will be automatically considered as trusted. I strongly recommend you to put this one to
ForwardX11Trusted is set to
ssh command will offer you to select the trust level for your session:
-X command-line parameter will consider the client as unsecure, this is recommended way to go.
-Y command-line parameter will consider the client as secure, this is only needed for very specific use-cases and should be used with care since you must consider that the remote applications will have a full access to your system.
However, as I stated above, you should not rely on the security feeling provided by the
-X "secure" mode to export display from untrusted environment to your local X server. But this should be sane enough if you consider this Docker environment as a trusted environment.
GUI application isolation is a domain under heavy study currently. While every other of the Linux system has went through large hardening measures until now, I have the impression that X was nearly always left appart, too large and complex to really evolve and be adapted to current needs, but leaving now a nasty weakness.
Secure platforms like Qubes OS which is precisely relying on a set of isolated Xen VM exporting X display to a Dom0 domain went through the pain to develop their own X protocol "reverse-proxy" between the X clients and server to ensure the server's security. I do not know any standalone equivalent of such technology, but you may find interesting ideas here: Best method to sandbox X applications in ubuntu. X server is leaving his last times and ought to be replaced by a more modern and secure alternative as soon as such alternatives become ready (see Wayland and Mir for instance), but until then at least check to who you entrust your desktop!