I understand the LDAP databases are more secure, but is it always necessary to use LDAP instead of MySQL for usernames and passwords?
I'm not sure I agree that "LDAP databases are more secure". After all, an LDAP server is basically a database server, with exactly the same security risks. LDAP is nice if you have the needs (software that can authenticate against LDAP, etcetera) and tooling, but security wise I don't see a difference between using LDAP and MySQL (given that you don't do stupid things like cleartext passwords or unsalted hashes).
Privilege Separation Principle
This is a classic case where isolation of your content database from your user database provides a security advantage.
- The Application Server knows the users password for the short time until it is compared against the password in the LDAP Server, where it is discarded and only the session and username information is kept for the life of the users session.
- The LDAP Server validates the password and delivers the user information to the app server.
- The Content Database has less critical information accessed through the app server.
The LDAP Server is not more secure, but it has a limited purpose, and thus the surface area for attack is much lower, as opposed to the typical user table stored in a content database which can be retrieved directly in the case of a injection exploit.
The LDAP Server and the Content Database are hidden (in some architectures) by the Application Server so that direct access is not required and only pass through attacks are possible, further reducing the possibility of compromise.
These multiple services do introduce a complexity that has costs and issues, but they constitute a pattern that encourages/forces the designer/developer to keep separate these specific accountabilities/concerns rather than collapsing the implementation into an insecure system.
Another way of thinking about it is that mistakes are always made when designing / developing / deploying a system, and without this separation of accountabilities, mistakes made in one domain can bleed into and compromise the other domains.
Note: Through a quick reading of the LDAP protocol, a password is a field in the LDAP database, so if that field is hashed (best practice) then the LDAP database doesn't need to know the plaintext password, and as a commenter notes there are special API's for avoiding issues with password authentication.
It is debatable that LDAP is more secure than MySQL.
I don't really see the benefits of adding yet another complex service to an infrastructure. If you want to Isolate authentication why not just use a secure (SSL), Isolated MySQL database. If you want a standardized way to access an authentication database, why not create a standard MySQL schema ?
LDAP is great when you have to authenticate a lot of different services against one single Backend (One Password for all or even SingleSignOn) or when you have different physical places scattered over the globe you have to authenticate against a single backend as LDAP is great for sharding informations.
But when that is not a MustHave and you are only thinking of using LDAP as authentication-backend I'd always vote for minimizing the systems which would mean If you need a database for your application then use that database also for authentication.
As an alternative I'd consider using OAuth or OpenID so users can log in using a different service and you do not have to consider Hashing passwords as you do not store passwords. That way you'd delegate authentication to an external service.
The problem of encrypted passwords is that most of the time the user sends her password unencrypted (only encrypted by an https-connection) to the server. From there the server can then use encryption or not to connect to the LDAP. But when you use the default bind command of LDAP you have to provide the unencrypted password to the LDAP so there is no benefit. On the contrary, when you use a database, you can store the password encrypted in the database and can then query for the encrypted password so between your application and the database the password is only send encrypted. That however does not encrypt the password between user and application!
Either they'll use a proprietary technology which is then irrelevant to your, or they use solutions on the market. What's important perhaps in your question is scalability of these solutions. For "LDAP Server" there is OpenLDAP which can be configured to use MySQL as backend storage. If I would need scalability I would look for this combination.LDAP is many times faster than SQL. LDAP is not a relational database. In fact, it's not a general-purpose "database" at all, it's a tree-structured directory.
A lot of the concepts you're familiar with from relational databases don't really apply to LDAP. For example, there are no "tables" and there's no "join" operation.