If you want to use a certificate installed in the Windows certificate store you need some kind of connector between the CryptoAPI provided by Windows and the PKCS standard used by Firefox. One example is P11CAPI that was originally intended for smartcards but should work for any installed certificate in the Windows certificate store.
Once you download the DLL that matches the bitness of Firefox you use follow the instructions in the above link (paraphrased in part below) to use the Windows certificate store in Firefox:
- Inside Firefox, go to Options → Privacy & Security → Certificates → Security Devices → Load
- Enter a module name (I recommend "p11-capi") and browse to the .DLL file.
- Click OK enough times to close out the various dialogs that have been opened.
Edit: From v72 on: "Experimental support for using client certificates from the OS certificate store can be enabled by setting the preference