Adding certificate that resides on smartcard to the Microsoft store with private key ownership

Question

How can we solve CryptAcquireCertificatePrivateKey fails with 0x8009200B when adding certificate from smartcard to local user store?

In an enrollment system where users generate smartcard certificate request online to a CA, the certificate is loaded 'offline' in the smartcard, for example several days after the request was issued so the certenrolllib objects used for the creation of the request cannot be used for installing the certificate on the card and the card generated the private key which will never and cannot anyway be exported outside of the card.

When we load the certificate in the card, we use the Minidriver API, we have the name of the key container that was used for generation of the key (usually a GUID, something like lr-e46f1586-7133-4284-895d-557e2261c24d) we start to read the cmapfile and we get the keycontainer index XX that corresponds to that GUID and we create a kscXX certificate in the mscp folder in the card minidriver filesystem, we compress with zlib the DER binary certificate obtained from the CA, we add a header and we load that on the kscXX file.

We can see that the on card certificate is well formed and that we can see it using various tools. The problem is that this certificate does not appear on the Microsoft user store. In fact it 'may' appear sometimes especially if the container is the default container, but anyway not immediately and not in all OS.

We created a lib using MSCAPI that get the certificate context and that proves private key ownership by using CryptAcquireCertificatePrivateKey in the following code (used in say: addCardCertToStore.exe tool for example)

Logging with PIN, getting the userkey etc.:

 fStatus = CryptGetKeyParam(
    hKey,                  // HCRYPTKEY hKey,
    KP_CERTIFICATE,        // DWORD dwParam,
    pCertBlob,             // BYTE* pbData,
    &dwCertLen,            // DWORD* pdwDataLen,
    0                      // DWORD dwFlags
    );

if (!fStatus)
{
    return 1;
}


pCertContext = CertCreateCertificateContext(
    PKCS_7_ASN_ENCODING | X509_ASN_ENCODING,
    pCertBlob,
    dwCertLen);

    //trying to prove privatekey ownership by calling CryptAcquireCertificatePrivateKey


HCRYPTPROV_OR_NCRYPT_KEY_HANDLE  hCrypt;

    DWORD  hInfo;

fStatus=CryptAcquireCertificatePrivateKey(pCertContext,0,NULL,&hCrypt,&hInfo,NULL);

     if (!fStatus)
{
     return 1;
}



    hStoreHandle = CertOpenStore(CERT_STORE_PROV_SYSTEM, 0, 0, CERT_SYSTEM_STORE_CURRENT_USER, L"My");

// saving certificate context to store

If CryptAcquireCertificatePrivateKey is not called, then the certificate is always exported to store, but without private key ownership and then cannot be used for operations like signature etc.

However, when we use CryptAcquireCertificatePrivateKey indeed '50% of the times' we can save the certificate to store with private key ownership, but not always, in several case we see 0x8009200B error (CRYPT_E_NO_KEY_PROPERTY , e.g. 'Cannot find the certificate and private key for decryption.') when call CryptAcquireCertificatePrivateKey. This error may happen for example the first time we launch the addCardCertToStore.exe then if we restart addCardCertToStore.exe a second time it may work, CryptAcquireCertificatePrivateKey returns OK, and certificate is installed in the store, sometimes we have to call it three times, sometimes it seems it will never complete and sometimes it works after we reinsert the card, sometimes it fails because the certificate was installed automatically after a while in the store and by deleting this certificate CryptAcquireCertificatePrivateKey works and add again the certificate on the store.

I know that the CSP is maintaining a cache and that it scans periodically the card to update that cache and that it may or may not detect a new certificate on card and add it to the store alone and that some MSCAPI operation can have the effect of having the CSP updating the store from the card certificates independently of the operation itself.

At the end the question is: why such a strange behaviour of CryptAcquireCertificatePrivateKey and how can we interpret the error code 0x8009200B (access denied) in that context?
And other point is that should we install the DER certificate on card not by using minidriver API but by using MSCAPI functions (if this is possible)?
How can we have a steady way of adding the card certificate to the store?

Answer 1

These things can be a bit complex because there are many pieces to the puzzle, and caches everywhere.

The theory is that you should use CertEnroll (provided by the OS) and let it do all the magic. CertEnroll can perfectly well support a long delay (even several days) between request generation and certificate import; however, the request generation and the subsequent certificate import must be done one the same machine, with the same account. The reason is that CertEnroll keeps track of an ongoing enrollment operation by saving a copy of the request itself in a dedicated store (outside of the card) which allows it, when the certificate comes back, to locate the private key and set all links.

(Theoretically, if the operation is done as a "normal user" on his stores, not the local machine stores, then the request should end up in the "roaming profile", thus removing the need for using the same machine; only the condition on the account is maintained. This would require some tests, though.)

Practice may differ for a number of reasons. Maybe the certificate installation is done on some dedicated hardware system, for instance. If you cannot use CertEnroll, or if some cache gets it wrong (it happens), then you must consider the following points:

• Link from certificate to private key is something which happens on the Windows side of things. When you call CryptAcquireCertificatePrivateKey(), it actually follows a property of the certificate context, i.e. the Windows-side object, to locate the CSP and container name within that CSP. Internally, that property consists in the CSP name and the container name. You can set it with CertSetCertificateContextProperty(); use CERT_KEY_PROV_INFO_PROP_ID.

  Such a property is normally persistent (it is written in the store structure, normally some files deep within the user's roaming profile) but you can keep it in RAM only with CERT_SET_PROPERTY_INHIBIT_PERSIST_FLAG. When the property is set in RAM only, then it will impact only a given CERT_CONTEXT structure, and will of course be limited to a single process.

• On the card, the relationship between certificate and private key is kept differently, with the cmapfile and kxc* files, as described in the MiniDriver API specifications. When you insert the card in a Windows system, that system is supposed to inspect the card for certificates, and push them into the local user's store, and set the links to private keys. The "certificate propagation service" is doing that, so make sure that it is started on your system. Otherwise, that automatic extraction of certificates won't work.

  The "certificate propagation" is triggered when the card is inserted. If you modify the card contents, especially when doing so "directly", then you must instruct the user to unplug/replug the card. The only other method I am aware of to trigger certificate propagation is to stop and restart the certificate propagation service, which requires administrator privileges.

  Certificate propagation is asynchronous and can take some substantial time (up to one minute or so), especially in conjunction with Remote Desktop and when several USB card readers are used simultaneously.

• There is a cache. The "smart card service" (SCardSvr.exe) maintains it. The cache is RAM-based, but the service writes it into a registry when it stops, so the cache resists reboots (the registry key is HKLM\SOFTWARE\Microsoft\Cryptography\Calais\Cache\Cache).

  The cache is meant to speed up things: reading data from the card incurs slow I/O. Certificates, container names, even public keys are stored in the cache. It is crucial that the cache is kept synchronized, otherwise various problems will occur, which are compatible with the symptoms you describe (you can also get others, e.g. NTE_BAD_SIGNATURE when computing a signature, because the BaseCSP will want to verify a just-computed signature, and will do so with the public key from the cache, not from the card).

  Whether the cache is considered up-to-date depends on the contents of the cardcf file on the card. This is described in section 5.4.3 of the MiniDriver API. Basically, that file will contain two 16-bit counters, one for the container contents (i.e. the public keys), another for the files. If you modify any container (e.g. generating or importing a new key pair) or any file (and you do !), then you MUST increment the relevant counter(s) in cardcf. Normally the BaseCSP does it automatically when needed, but if you reinitialize a card, or modify it eternally, then you may have to do it manually (I have personally witnessed some bugs in the SCardSvr cache, too, so I ended up forcing a cardcf update with card-specific APDU commands).

  Alternatively, you may try to configure the card so that it reports that no cache should be maintained for its information; lookup CP_CARD_CACHE_MODE in the MiniDriver API. Not all cards support modifying the cache mode, though.

• Compounding the already complex situation is CNG. This is a brand new cryptographic API. There are some bridges between CryptoAPI and CNG; e.g. the CRYPT_ACQUIRE_ALLOW_NCRYPT_KEY_FLAG for CryptAcquireCertificatePrivateKey(). When setting the CERT_KEY_PROV_INFO_PROP_ID property, you may want to replace the CryptoAPI BaseCSP name ("Microsoft Base Smart Card Crypto Provider") with the corresponding CryptoNG name ("Microsoft Smart Card Key Storage Provider"). This may confuse some applications which are not aware of CNG.

  Playing with CryptoAPI and CNG can be made necessary with regards to card PIN handling. In order to ease user operations, both CryptoAPI and CNG maintain an in-memory, per-process cache of the user PIN. However, the two caches are distinct ! Moreover, with a recent card (i.e. one which supports MiniDriver v7, as opposed to MiniDriver v5), there are "session PIN", automatically generated, and the caches will keep a copy of the session PIN, not the PIN as known to the human user. If two subsets of your application use the card, one through CryptoAPI, the other with CNG, then they will compete for the PIN, each one triggering the generation of a new session PIN which invalidates the session PIN for the other one. The consequence is repeated "PIN entry" popup thrown at the user, who is rarely happy about it.

  In particular, Internet Explorer on Windows 7, and more generally the SSL client code, when accessing the private key for certificate-based client authentication, tends to force CNG use. If you use the same key, within the same application (e.g. an ActiveX control loaded in IE), then you should use CNG as well -- otherwise, the PIN battle rages.