This question is likely to be country-specific. In the US, an account number is generally not enough to steal money from someone's account. This is fortunate, because every time you write a check or make a bank payment to someone, they receive your account number.
However, learning someone's the bank account number is enough to learn their account balance. There is an attack that is not widely known:
- Most banks have a phone number that merchants can call and, via an automated voice response system, learn whether a particular account has enough money that a check for a particular amount will clear. Basically, you just call up, hit a few digits to go through the phone tree to the merchant check verification option, then type in the account number and the amount, and the phone system will respond with whether the account balance is at least as much as the amount you've provided.
This allows an attacker who knows your account number to learn your bank account balance, by using binary search. This is a confidentiality breach that is not widely known.
If this bothers you, you may be able to protect your own account against this by calling up your bank and asking them to put a fraud alert on your account. At least for my own bank, when the bank does this, it disables the merchant check verification service for your bank account.