I am doing a project with an architecture of single server and multiple database. Every client request will contain a header containing database name that this request should connect. Is this a good idea for disclosing the database name ? Is this a security issue ?
This is not a good idea. An attacker could change the database name, supply an invalid one, or put special chars on the database name and possibly exploit a SQL injection. It's possible to do a lot of damage depending on your setup.
You could use server-side sessions, and send only the session token to the user. The database he is connecting to should be stored on the server side session.
It is a security issue, and tampering with headers, or really, anything that you have on the client side is very easy. You should never trust anything that coming from the client.
I would definitively suggest that you have a white list of acceptable values that is enforced on the server side to avoid SQL injections. This list could be generated dynamically, but make sure that you never, ever, use something that comes from the client in a database connection string or in a dynamically constructed query.
As other have suggested, the database name should be stored on the server-side, in a session token, and not sent through HTTP headers.
OWASP have loads of good information on the subject of SQL injections. See https://www.owasp.org/index.php/SQL_Injection_Prevention_Cheat_Sheet
If all you are passing over is a simple name which identifies a database to use, then I can't see any immediate problems (provided you do some server side validation).
I would try to avoid using names that give away more info about the database than needed though (e.g. server addresses, system- or version-info, etc). In fact, it might be better to just use a numeric ID or something similar to identify the servers instead. Map this to the actual databases on your server, where the chances of anybody seeing this info is smaller. The less info a client needs to know, the better!
Also, you should verify a user's right to access whichever DB is specified. If a request from a client is manipulated to attempt to access a DB other then the one it should, then logic on the server side should cancel the request, or at the very least verify that the user is in fact allowed to access this other database.