Yes, CVE-2014-0196 can be used for privilege escalation, because it allows overwriting buffers in the kernel outside the one allocated to your pty.
This means that a non-privileged user could use it to become root. It's not limited to the friends that you've allowed access, either. If an attacker were able to compromise a service running on your system (a webserver, for example), they'd be able to escalate to root as well. Even just by finding a vulnerable webapp (PHP script, python app, etc.) that allows execution of local commands, they could escalate to root.
There's a PoC for specific kernel versions here: http://bugfuzz.com/stuff/cve-2014-0196-md.c, so it's definitely doable, though the PoC given may not generalize to all affected versions of the kernel.
Chroots would have no impact on this exploit, as it requires only being able to make some syscalls, and chroots are about limiting access to the filesystem. An attacker would need to either compile their exploit locally or be able to download it, but it could even be transferred over an SSH or similar connection, so unless there's nowhere mounted writable and executable to the user, the chroot offers no protection at all.