How does StrongSwan enable different crypto libraries?
By default, strongSwan uses its own implementations for cryptographic algorithms. These are provided by e.g. the aes, sha1 and sha2 plugins. The default implementation for public key cryptography (RSA, DH) is provided by the gmp plugin, which relies on libgmp.
Enabling the openssl or gcrypt plugins, while not-building or not-loading the default plugins, allows to replace the default implementations with those provided by the respective libraries (libcrypto/OpenSSL or libgcrypt). The actual features provided by these plugins depend on how the underlying library was built, for instance, if OpenSSL was built with support for Elliptic Curve Cryptography (ECDSA, ECDH).
You can check which features each plugin provides with
ipsec listplugins after starting the IKE daemon with
ipsec start. The currently used implementation for each cryptographic primitive can be listed with
ipsec listalgs (this depends on the order in which plugins are loaded, the first implementation will be used).
It's important to note that these plugins are only used for the key exchange (IKE,
ike keyword in ipsec.conf). The ESP traffic is handled directly by the operating system's IPsec stack, for instance, the Linux kernel, which provides its own modularized crypto framework.
The parameters you may use in
ipsec.conf depend only on the availability of a specific algorithm, either provided by any of the plugins (IKE) or by the operating system's kernel (ESP). All supported algorithms/keywords and their availability in plugins and (Linux) kernel can be found on the strongSwan wiki.
Unless you've got a pressing reason to, I'd suggest that if you're trying to configure something like an IPSEC tunnel for the first time, stick to default values as much as possible, it'll make the process easier.
That said if you need to use gcrypt with strongswan, they have configuration examples for that scenario on their site here