This is a issue of who controls the code. Let's leave out that there are some modern framework like node for JavaScript on the server side.
The languages themselves are not necessarily more vulnerable. Most flaws are not results of the underlying language, but of the various logic checks, input sensitization, etc.
That said, you cannot control the code client side, which means that the local user can modify it. It can also be modified by man-in-the-middle attack on the network or by a man-in-the-browser attack where the attacker has compromised the local machine/browser.
On the server side, they would have to break into your server to modify the code. However, that does not mean the code is necessarily more secure.
While you can modify the JavaScript through your code inspector locally, if you did something malicious, you would be doing it to yourself. Just because you change the code doesn't mean it will affect other users. However, you can create attacks against the page request that you then send to someone else which will result in XSS or CSRF. This would result in an attack being carried out against the user. At the same time, you can change the request so that malicious requests or data are sent to the server which result in a buffer overflow or a code injection (e.g., SQL injection).
One of the differences is that you can more easily craft an attack if you have access to the code. Since you can see the source of the HTML and JavaScript, it may be easier to craft an exploit, but that same knowledge might lead to crafting a SQL injection.
Now, to your last point. You should ensure all of your security checks are performed on the server side because you cannot trust client-side validation since it only affects what's going on in the browser. An attacker locally or MiTM can still change the raw HTTP traffic, which would result in your client-side validation effectively being ignored. Client side validation is a convenience for your users and to save some cycles on your server.