What does it mean to be FIPS 140-2 compliant?
It is a security standard provided by the US Government and a requirement when working with the government. Products that meet the standard are certified and audited to meet certain requirements, partially which crypto libraries they use. Devices that have this standard are not guaranteed to be bug-free but have some level of the audit to them.
Is it enough to use FIPS 140-2 algorithms?
It means that your library has the potential to meet the standard if configured correctly. And can make audits easier as you can point to compliant libraries.
Or should we get a FIPS 140-2 approval?
Who is your customer base? Does this base need the standard, or would it allow you to move into new desired markets? Notice Windows support the standard, but it is not enabled by default. FIPS tends to add restrictions to your design choices and recommend older, more battle-tested algorithms, so using it will limit your ability to use newer major break-through if needed or desired.
Should we receive the FIPS 140-2 approval for each component?
Or enough to receive it for OpenSSL? Or should we receive the approval
for each library that handles cryptography?
The process will vary on the product created and usage. It's best to talk with the audit company about that. Many should be able to give you an idea of the process before you need to pay them.
What happens if we update a library? E.g., we upgrade OpenSSL or Bouncy
Update or changes to the product may require recertification, and best to talk to the auditor.
Should we receive the approval once again?
Not sure what ask is here, but I would get if required, or you feel it would increase a goal enough to justify time/expense. In general, audits are not fun.