We are currently developing a security software for Windows. This application consists of a service running in user mode and a driver in kernel mode. These two need to communicate, so that the service can get data from the driver and send actions to perform. The communication is implemented using IOCTL.
The problem is, how we should secure this communication, so that no 3rd party software can abuse this.
From the documentation I've learned to set permissions to the device-file, but these are very limited. We can only grant permissions to the admin-group or the SYSTEM account.
The service itself does not need elevated privileges and we therefore don't want to run the service with too much rights (principle of least privilege...)
Our approach now is to create a local user with restricted privileges that runs the service, like it is often done in Linux.
We can also get the user from the IOCTL calls and by comparing them to the expected user, we are able to 'grant' and 'deny' access.
The question now is, is this approach secure? Or does it have flaws we do not see? Because we could not find anyone following this approach.
--edit--
Considerations we have taken (assuming the user's password is secret): as a normal user you cannot pretend to be the app-defined user and therefore you can't communicate with the driver. An Admin/SYSTEM has this possibility (at least the password can be changed). But an Admin/SYSTEM could also communicate with the driver if the permissions on the device are set.
There are ELAM (early launch anti malware) drivers that can launch protected processes. This would probably solve the problem, but right now we would not get the required certificates from Microsoft.