I am new to implementing information security. However i am trying to harden a windows server.I am using regedit to create parameters like SynAttackProtect,TcpMaxHalfOpen and TcpMaxHalfOpenRetried and set values to them. But i am not sure if this is a good way to do it. As in i browsed to this location HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\TcpIp\Parameters and manually adding these parameters. Kindly suggest if this is okay
Completely preventing TCP SYN flood attacks is impossible; the best you can do is to mitigate them. Those registry settings are a valid option and may help you if you are under a weak or mild SYN flood attack. They work by simply limiting the duration and amount of half open connections when undergoing the symptoms of a SYN flood. In other words it makes the server less patient in waiting for the client to complete the three-way handshake (i.e. completing the connection).
Here are a few other common defenses to help mitigate SYN flood attacks:
Increasing the server(s) resources (e.g. adding more memory, or more servers for load balancing)
If filtering IP addresses is an option, this would be the most effective approach. For example, if you know your website will only be used by a certain IP address range (or geographical location) you could filter the IP addresses that are allowed to connect. A white-list of IP addresses of known legitimate clients could also be used to give higher priority to those connections.
You could also check out SYN Cookies which is a creative mitigation method that could help as well.
For additional reading on these SYN flood mitigation methods, I suggest reading section 3 of RFC 4987
Hope this helps!