Agree with Jeff Ferland, database servers should be on their own: you should have a clean network for replication & backup.
Pardon my ASCII art, a quick overview of a reasonable ideal:
[lan]--+ +-- [database-zone]
- Run a reverse-proxy, Apache+mod_security/varnish/nginx/WAF/whatever, in the proxy zone. Add load-balancing/failover here if needed too. Also add proxy/relay servers for outbound connections (DNS, SMTP, HTTP proxy), if required.
- When application logic runs on a web server (Java/PHP/ASP), I prefer to call it an application server.
- When you need to scale you can scale horizontally, load balancers make this easier. You may also consider replicating static and/or unauthenticated content to the front-end proxies (advance thinking about URL paths, hostnames and cookies pays off here)
- you might want to add one or more zones of type: IDS, management, backup, remote access, outbound proxy
You're trying to mitigate, so:
- inter-zone communication must be limited to the minimum required for services, administration and monitoring purposes.
- a reverse-proxy accepts untrusted connections from the internet, and can only initiate connections to services on application servers, no outbound internet connections. If you want to classify (color) your zones by nature/traffic you need to consider carefully termination of HTTPs, and if you want to create new HTTPs connections to the app servers (update: you probably do, but you can configure a small set of fast ciphers, and omit CA bundles to keep overheads low).
- application zone accepts semi-trusted connections from proxies, and can initiate connections only to databases. You can trust your application servers a little bit more when you know they're not talking directly to the internet.
- database servers accept connections only from application servers, the database zone should be your "cleanest" network
- consider using different firewalls (vendor/product) for the outer- and inner-firewalls
- for required outbound services (DNS, SMTP or patching/updates) these should go via a distinct server (e.g. on the proxy-zone, or seperate outbound-proxy-zone).
- same goes any outbound CC validation HTTPS connections. (If you're unlucky enough to have some vendor or services black box for validation, these ought go on a dedicated zone too, IMHO.)
- use public IP addressing only in the proxy zone, private addressing elsewhere. No server outside the proxy zone(s) need have a public IP, NAT, or even a default route to the internet.
Separate zones makes your IDS's job easier, and logging more effective. Zones should be layer 2 (switching) and layer 3 (IP) isolated.
If you have the resources, add a management-zone, separate management NICs for each server (protected ports if you can). Service monitoring is best done over the operational network.
In reality you may end up compacting the "ideal" network to a single firewall and VLANs. Any dev/preprod environments should copy the operational network as closely as possible (especially if you don't want to chase your tail debugging issues introduced, like dropped idle connections). If you consider your options now with the above in mind it should be easier to migrate in future, i.e. shortly after the next visit from your friendly neighborhood PCI-DSS auditor ;-)