Although you didn't say so, I'll mostly assume openssl. Lots of software uses PEM format for certs (which is base64 as you showed PLUS the dashes-BEGIN and dashes-END lines, those ARE NEEDED), and quite a bit uses a file containing multiple PEM certs called a bundle, but AFAIK only openssl, and apps calling it, uses a PEM bundle file for trusted CAs.
For info, the length of the base64 lines differs because of following different standards. The PEM RFCs which first defined base64 said 64. The MIME RFCs which are now more widely used say 76.
Option 0: don't bother. An openssl "CAfile" works just fine with the same (CA) cert occurring more than once. It takes a little more memory, but on modern desktop (or higher) systems you probably can't even measure the effect of less than a few thousand extra certs.
Option 1: canonicalize. Filter each cert through
openssl x509 with all defaults (which is copy, inform PEM, outform PEM) i.e.
openssl x509 <oldfile >newfile. The result will consistently use 64-char breaks. As a bonus, you can specify
-subject to add a (fairly) readable name at the beginning of the file -- which openssl or apps using it will ignore, but other apps might not.
But for both 0 and 1, if you have different certs for the same CA, such as an expired one and a renewal, or a broken one and a fix, which aren't duplicates by your definition, that will NOT work reliably; openssl to date (1.0.1) uses only the first cert found for a CA (issuer) name and AKI if available, which if you aren't careful is the wrong one. (1.0.2 is announced to have changes in cert chain validation, and I haven't looked at the details yet, so this may change.)
Option 2: use name hashes. Use the openssl "CApath" or "CAdir" method in addition, or instead: create a directory in which you put each CA cert as a file, and everytime you add one do
c_rehash (on Unix, or Windows if you install perl, or else a BAT-ified equivalent). If your new cert and any existing one have the same hash of subject name you will get links (or names) with a suffix other than ".0". Two certs with the same subject hash e.g.
abcdef01.1 ALMOST certainly (not quite) have the same Subject i.e. CA name, but may or may not be the same cert; compare those specific files.
Note that having constructed a "CAdir" in this format, openssl commandline and at least many apps allow you to use it instead of "CAfile"; some even give the mostly-outdated advice that it's "more efficient" (in 1995 it made a difference, today it's probably 0.1 msec instead of 0.2 msec).
Other tools: if I've guessed wrong about openssl, you can do the canonicalize option by importing to and exporting back out from the Windows keystore, the Firefox keystore, or a java keystore. I expect Mac, Apple, and Android have similar capabilities but don't know them.
The first two have convenient (but tedious) GUIs; for Windows I believe there is still a commandline form but, typical for MS, they change it everytime you blink so I stopped trying to keep up. These also have the property, possibly bad or possibly good, that if you leave these certs in the truststore and don't delete them, your respective browsers (IE and maybe Chrome, or Firefox) will trust website certs from those CAs.
For java (assuming JRE in your PATH, else use pathname)
keytool -keystore whatever [-alias x] -importcert -rfc -file certfile
keytool -keystore whatever [-alias x] -exportcert -rfc -file newfile
or even easier: if you import and leave in one JKS all the same certs as your PEM bundle, each with a unique
-alias you must specify, and you try to add another copy of a cert already in the JKS, it will warn you. It won't detect certs with the same CA, however.