One of the reasons that banks often have case insensitivity in their passwords is because of phone banking: banks existed FAR before the internet existed, even before telephones were a thing. So once telephones became widespread, many major banks allowed people to to banking stuff via the telephone. it makes sense: all you need is two account numbers and a code to verify that you're the one doing the transaction. For this code, you usually went to the banking institute.
However, since you needed to enter your code using the number pad on your phone, the system just responded to the number presses, not the actual password. That means that there wasn't even a distinction between lower and upper case, because there was no difference in how you entered them on a numpad.
Once internet banking arrived, those systems used a similar backend to the phone banking system, including using the same passwords so users didn't have to remember extra passwords. However, this lead to the problem that it was trivial to make the difference between a lower and uppercase letter, and the way the passwords were entered in the system during the phone banking era was inconsistent: some tellers would use capitals, some would use lowercase, some would use CamelCase,... To prevent people from having to return to their bank to clarify this, they HAD to make passwords case-insensitive. Note that this part might not be applicable for all banks, but some banks have this reason.
https://en.wikipedia.org/wiki/Telephone_banking - Wikipedia article about telephone banking;
https://www.ing.be/en/retail/day-to-day-banking/self-banking/pages/phone.aspx?tabName=Details - Article on Belgian bank website about phone banking;
http://www.hsbc.co.uk/1/2/ways-to-bank/phone-banking - Article on major British bank about phone banking.