I'm pretty sure a lot of you are familiar with the "USB Rubber Ducky", the USB stick that emulates a keyboard. Is there software for Windows that will prevent this type of attack, while still letting me use legitimate keyboards? For example, is there a program that will prompt "Please enter the words you see on the screen to activate your keyboard" when you plug in a new keyboard, sort of like a CAPTCHA? If not, is it easy to write such software?
I'm not aware of any built into windows. However there are third party device control products that lock the screen when a new keyboard is added. This thwarts some of the "rubber ducky" attacks by preventing a simple set of keystrokes from running immediately on insertion of the usb device doing anything useful.
I'm not aware of any such software, and in general, such software would likely be ineffective and potentially harmful. Consider the following scenarios.
First, one could create a real USB keyboard that contains a special chip that, upon activation with a special key sequence or upon remote activation, sends malicious text, but otherwise behaves like a normal keyboard. That wouldn't be noticeable with your proposed software and yet would still be malicious.
Second, one could just create a USB keyboard that logs all keystrokes and uploads them via a cell connection. Such devices already exist, and while they wouldn't grant someone access to your computer, they would allow someone to just compromise all of your accounts remotely.
Third, many common security keys like YubiKeys have an OTP mode where, upon pressing the button, the key sends the one-time password, including an Enter key. These look and act like USB keyboards. This is a secure and valid use of automated USB keyboards, and while it's better to use something like the FIDO2 mode when possible, making it impossible to use these (since they would be unable to enter an arbitrary challenge), which might harm security.
Thus, it's unlikely that such a device could be effectively prevented from activation or exploitation and it's better to avoid untrusted USB devices altogether. If you can't do that, it would be better to pass them through to a restricted VM running a throwaway live CD Linux distro instead so their damage is limited. Note that doing so doesn't prevent things like physical damage to the port, though.