How to use multiple smart cards with gnupg?

Question

I have two smart cards: SCard1 and SCard2. The same keys are stored on both of them.

One of the feature I use the most is SSH authentication to access multiple servers. But I cannot use the second one after having used the first both of the smart cards (not simultaneously) on the same computer.

The following message is displayed:

Please remove the current card and insert the one with serial number
[a long ID containing the SCard1 identification number]

Steps to reproduce the issue

Let's prepare a clean environment.

$ export GNUPGHOME=/tmp/gnupg
$ mkdir /tmp/gnupg && chmod go-rwx /tmp/gnupg
$ curl [public key URL] | gpg --import
gpg: keybox '/tmp/gnupg/pubring.kbx' created
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  6194  100  6194    0     0   2408      0  0:00:02  0:00:02 --:--:--  2408
gpg: /tmp/gnupg/trustdb.gpg: trustdb created
gpg: key 3D5DDE26A0B55831: public key imported
gpg: Total number processed: 1
gpg:               imported: 1

With SCard1 plugged

Then I insert my SCard 1.

$ ls /tmp/gnupg/private-keys-v1.d/
$ gpg --card-status
Reader ...........: Yubico Yubikey 4 OTP U2F CCID 00 00
Application ID ...: [SCard1 identification number]
blahblah
ssb>  rsa2048/467299D324A21B24  created: 2017-01-10  expires: 2018-01-10
                                card-no: [SCard1 identification number]
ssb>  rsa2048/11771A79F4B85B41  created: 2017-01-10  expires: 2018-01-10
                                card-no: [SCard1 identification number]
ssb>  rsa2048/F31D41149A84B062  created: 2017-01-10  expires: 2018-01-10
                                card-no: [SCard1 identification number]

The last key is used for authentication.

$ ls /tmp/gnupg/private-keys-v1.d/
somehash1.key somehash2.key somehash.3.key

I guess those hashes come from the keys I've stored on my smart card.

$ gpg -K
/tmp/gnupg/pubring.kbx
sec#  rsa4096 2017-01-10 [SC] [expires: 2019-01-10]
      [primary key ]
ssb>  rsa2048 2017-01-10 [S] [expires: 2018-01-10]
ssb>  rsa2048 2017-01-10 [E] [expires: 2018-01-10]
ssb>  rsa2048 2017-01-10 [A] [expires: 2018-01-10]
$ ssh example.com hostname
*** asked to enter pin
example.com

Everything works as expected.

With SCard2 plugged

Let's unplug SCard1 and plug SCard2.

$ ssh example.com hostname

And the following message is displayed:

Please remove the current card and insert the one with serial number
[a long ID containing the SCard1 identification number]

Please note I've got the same result when I insert first SCard2, and then SCard1.

Part of a solution

Some searching on the Internet made me understand that the "key stubs" are stored in ~/.gnupg/private-keys-v1.d/. Deleting the whole directory solved the problem. Maybe just deleting one of the .key would do the trick.

The question

Is it possible to use both of the keys without deleting any file?

Answer 1

With gpg --with-keygrip -K you will get the keygrip identifiers for your keys. These ids correspond to the keystubs in the private-keys directory.

In case you have 3 subkeys (encryption, signing and authentication) you will have three files in GNUPGHOME/private-keys-v1.d (~/.gnupg/private-keys-v1.d on linux).

When using different smartcards with the same keys on it, the identifiers will be the same, but the binary contents of the file will differ.

Deleting is an option, but you might inadvertedly delete other private key stubs as well.

Another option is to make a backup and switch them out every time you switch cards (<id>.key.card1 and <id>.key.card2 will be copied to the current/active <id>.key file).

The best option to do this seamlessly is to use udev as suggested by @Jens Erat. On addition of the smartcard, backup the current keystubs (<id>.key.<cardid>) and recreate (or copy back to <id>.key in case of an existing backup) the keystubs you want. Recreation is as easy as running gpg --card-status.

How to set this up with udev:

• run udevadm monitor --environment'
• now add the smartcard and watch the output so you can build a match
• build the matching rule (so it matches once) and RUN a script on addition.
• reference a script from the udev rule to make the magic happen (detach the script)

The next example is for the yubikey 5 in /etc/udev/rules.d/99-gpg-smartcard.rules:

    # /etc/udev/rules.d/99-gpg-smartcard.rules
    # helpful commands:
    # udevadm monitor --environment --udev --kernel
    # udevadm info -a -n <device-path>
    # udevadm test $(udevadm info -q path -n device_name) 2>&1
    # udevadm control --log-priority=debug
    # udevadm control --reload-rules && udevadm trigger
    SUBSYSTEM=="usb", ACTION=="add", \
    ATTRS{idProduct}=="0407", ATTRS{idVendor}=="1050" ,  \
    ATTRS{manufacturer}=="Yubico", ATTRS{product}=="YubiKey OTP+FIDO+CCID", \                                                    
    SYMLINK+="yubikey",  \
    RUN+="/usr/bin/at now -f /home/USER/bin/gpg-smartcard.txt"

        # Since nested quotes are not possible in RUN (outer double quotes
        # and inner single quotes) and cannot be escaped, we need to make sure
        # there are no nested quotes.
        # Furthermore, we need to detach our script from udevd for it to be able
        # to use the device (smartcard), since the RUN command
        # is blocking the daemon and the device is not yet ready, therefore we
        # are using the 'at' command to completely detach the job from udevd.
        # Since we need to use 'su' with the correct USER in the command
        # and multiple arguments (/bin/bash with the scriptname)
        # we cannot use the full command here when using 'echo' to 
        # pipe it to 'at'. 
        # So we are stuck reading the command from a textfile when using 'at'.
        # In the textfile is the following command:
        # su -l USER -c "/bin/bash /home/USER/bin/gpg-smartcard.sh"

in /home/USER/gpg-smartcard.txt

su -l USER -c "/bin/bash /home/USER/bin/gpg-smartcard.sh"

in /home/USER/bin/gpg-smartcard.sh:

keys=( \
  <ENCRYPTION>.key \
  <SIGNING>.key
  <AUTHENTICATION>.key \
  )
original=~/.gnupg/private-keys-v1.d
target=${original}/backup
mkdir -p ${target}
now=$(date +'%Y-%m-%d-%M:%S')
for i in "${keys[@]}"; do
  mv ${original}/$i ${target}/$i.bak.${now}
done;
sleep 1
# gpg --card-status will also work
# use 'gpg-connect' and then 'help' to find out the commands
gpg-connect-agent "scd serialno" "learn --force" /bye >> $log 2>&1

I have written a HOWTO for setting up your yubikey here

Sources:

• using udev explained
• using long running processes with udev
• using at with udev
• yubikey forum about gpg-connect-agent info
• gpg documentation for scdaemon

Answer 2

The new release should fix these issues.

Version 2.3.0 includes, among other things,

• scd: Improve support for multiple card readers and tokens.

https://lists.gnupg.org/pipermail/gnupg-announce/2021q2/000458.html

Answer 3

GnuPG stores the serial number of the card containing the key in the key's stub, and you can only setup one card per key.

For the same key, you only have one key stub -- thus deleting "one of them" will not help. The only thing you might have luck is by using UDEV to get notified of smart cards being plugged in, and switching or rewrite the key stubs to point to the provided smart card.

Answer 4

Its not clear from the question what you use your smart card enabled yubikeys for except SSH. But I would recommend skipping gnupg all together and use the PKCS11 support directly in OpenSSH instead. This will use a pkcs11 library and talk directly to the card.

If you do other stuff that require gnupg you can safely ignore this answer.

Answer 5

I have not yet tested my Yubikeys (primary and backup) with SSH yet, but I ran into a similar issue just setting up my backup for use with gpg. I followed the instructions from Simon Josefsson to create and load the subkeys to the card.

https://blog.josefsson.org/2014/06/23/offline-gnupg-master-key-and-subkeys-on-yubikey-neo-smartcard/

On my first attempt, I simply tried to move a copy of the same set of three subkeys to both cards in much the same way one would duplicate TOTP secrets to the backup device. After moving the subkeys to the first card, I restored the .gnupg folder from backup and repeated the keytocard operation to load same three keys to the backup device.

That approach failed. When I subsequently would run "gpg --card-status" to generate the stubs on the online machine, I would only see stubs generated for the first YubiKey inserted. gpg will not generate two stubs for the same key to point to two different devices. Upon performing any gpg operation with the backup key inserted I was instructed to remove it and insert the primary key.

What seems to be a simple solution is to create a 2nd set of subkeys (six subkeys in total) to load to the backup device. With that configuration, 'gpg --edit-key' shows the 6 subkeys each assigned to their appropriate device, and gpg functions operate correctly with either YubiKey inserted with no additional prompts to switch keys.