I have a corporate web portal which is used for staff collaboration including a web-based mail interface connected to MySQL Database, corporate calendar, file storage, customer contacts data with phone numbers and emails and all kinds of other data. Of course some of that data is sensitive as it relates to internal corporate matters as well as customer privacy.
The whole site is written on PHP+JavaScript with MySQL database on Ubuntu (Apache). I have chased up all the code and replace all SQL statements on Prepared Statements to prevent SQL Injections. None of the web forms allowed to users to submit HTML data, only pain text, I also don't store HTML into MySQL tables to prevent Cross-Site Scripting. I put a brute-force defense by delaying next attempt for a user after typing a wrong password. The authorization done in two steps with user first typing his/her name and password and then the system sends SMS to that user with an authorization code. The whole site works over HTTPS only and connected to internet through a firewall which permits only 443 port for incoming connections, i.e. no FTP, samba, no external MySQL connection or any other services are available. The main page of the site is empty and Search Engines can't even find a login page to index so if one doesn't know the exact address he couldn't even start an authorization process. I don't know of any other ways to hack it.
However I was trying to get a consultation from a security specialist and despite all the measures taken he insists to connect any external clients over VPN client (L2TP IPSec) which is integrated in any modern Windows or Linux Operating Systems. Since this is not a public site I could do that but that will take additional work with the clients to explain each one on how to connect and give them one more login-password pair. I was also told that PHP originally being a Personal Home Page language with no attention to security still might have vulnerables and security breaches. He couldn't give any examples. I wonder if those are really exists and what are they? Would I really increase the security by switching the clients to the VPN access instead of direct HTTPS or it will just uselessly bother them?
UPDATE (a month later)
After playing around with VPN tunnel and VPN clients I don't think it is a good idea to create a VPN tunnel over HTTPS because what I just have is WEB server with HTTPS behind NAT I have all ports blocked except 443 so the only think a user can do is to access that.
If I create a VPN channel I let users to access my local network so they have access to the NAT router from inside (from LAN ports) and they have access to the whole server which plays role of the WEB server. So a malicious user who hacked VPN login/password and got inside LAN i.e. got behind the NAT and he can now start hacking not only the server on all ports but also the router. To prevent that I need another firewall and probably another device to create virutal LANs.
Does it really worth it?
More on netwrok diagram
As to the netwok diagram the local server has two network interfaces: one for internal network and other for Internet connection.
The Internet going into that local server through two NAT points:
1) The Internet connection goes from the external IP through Windows Server 2008. That Windows Server shares the Internet connection to people in office and also it has RRAS set to forward port 443 to one of the local addresses (let's say it goes to 192.168.0.100). There is also 3rd party firewall allowing incoming connections only on 443 port.
2) Then I have a router on that local address (192.168.0.100) which can handle VPN tunnels of just port forward futher. For right now it only forward port 443 to its LAN which the local server is connected to. That makes the local server separated from all the other computers that is getting internet from the Win Server 2008.
Separately from that there is a whole independent network with no internet connection. The local server is part of it using it second network interface. There are no any routers there, only a switch.