I'm not a financial person and have wondered this as well. So, I think the best way to answer this question is by starting with which services a bank can provide and then what information is required to use those services. I investigated this by actually using the below services. There are two services that I know of to transfer money:
- ACH: Your bank will require you to validate the foreign institution transaction before you can send money. There are also daily limits. This is largely the one security barrier between your account being drained by ACH.
- Wire: Your bank sends your funds to wherever you tell them to (details below). There are no limits on this transfer and the bank will most likely assume zero responsibility in the event of loss of funds (at least mine did). The institution also does not verify the endpoint like they would with ACH; if you or the financial advisor fat finger the routing number or account number you are potentially at a loss. I asked both institutions before sending a wire transfer what happens if I miss a number and they both advised me not to do that and to triple check the numbers.
Both ACH and Wire require the same numbers to transfer: Routing number and account number (they did in my case, maybe not your institution, again you have to check). Wire transfers almost can never be reversed (I cannot back this up; but it seems like its in every liability). ACH however seems to have that capability. ACH again has limits; one institution I knew of it was $3,000 a day. This makes it next to impossible to drain your account via ACH, and since most ACH can be reversed it makes it less worth while to even try. Its also worth mentioning that ACH are limited to within the USA only. The only way to drain your funds to a foreign institution would be via an international wire transfer which has even more strict policies/regulations than a domestic wire. Also worth mentioning is that you usually need to know the account details of the foreign account to conduct the wire (such as the full name and address of the account holder).
So, with that being said. If you give your password to your bank account whomever it is will have access to your account numbers. This means they could setup an ACH transfer and disable any alerts you have and then setup a recurring ACH transfer that you may never notice. Doing a wire transfer is harder; most if not all banks require you to be physically present to conduct a wire transfer. Some do not; some allow you to fax in forms and your ID... So, let that sink in.
Another perspective for you is if you don't give them any information except your banks routing number and account number. They can then use ACH to request and take funds from your account (think eCheck or reverse Direct Deposit).
With that being said, you should never give your bank password or account numbers out. I find it disturbing that people will give these numbers out willy-nilly for gym membership (yes I'm pointing at you Planet Fitness!).
I should also mention that the routing number for the bank/institution is public information. You can often see it on the front page of the website; this means its one less number sequence for an attacker to find. The number the attacker wants is your bank account number.
You should read over the Security section for wires. Also worth mentioning is that the USA uses SWIFT to conduct wire transfers; this seems to be the target for malware and other types of attacks.