JavaScript is "relatively safe," but not "absolutely safe." Any code you run on your system has a potential to do harm. There is no perfectly safe system, except for the one that never used. JavaScript is safer than putting an unknown USB device in your computer, and safer than a binary you download from a shady website or get in a suspect email attachment, and much safer than some of the scripts you'll find on websites that tell you to copy-paste them in to your shell.
It has safety features: a sandbox to help isolate the process, a relatively limited API that has security constraints to help avoid running arbitrary computer instructions, and security controls meant to limit exposure of sensitive data like fingerprints or cross-domain data sharing. These are on top of your operating system's controls applied to the browser binary to limit bad behavior, and anti-virus applications that can help stop such attacks.
However, it is not absolutely safe. Bugs in the browser's runtime engine, the browser itself, the anti-virus, or even the processor itself can all compromise JavaScript's security. The system is only as secure as its weakest security. The security of JavaScript is mostly meant to prevent "casual" exploits (such as an 8-year-old learning JavaScript for the first time and accidentally writing an exploit), but doesn't stand a chance against dedicated attackers. JavaScript is complicated enough that there are bound to be bugs, perhaps in weird and unexpected ways.
Those experienced in hacking, pen testing, and security can, and do, browse through source code, debug executables, and do whatever else they can to try and find chinks in the armor. The weak points of the JavaScript implementation. And JavaScript is big enough that such chinks exist to begin with, since it is virtually impossible to even automate all the possible tests that would find these bugs.
Generally speaking, any typical script you might run in to on a typical website is probably "safe", especially those linked to by major search engines. However, once you start going off the beaten path, it is incredibly likely you'll get your system compromised at some point if there's even a single weak point. It only takes one really good exploit, or sometimes two or three in tandem, to completely take over a system.
As it is, only enable JavaScript for sites you trust (I personally use NoScript for this purpose), always keep all your software up to date, and always pay attention to browser warnings like invalid certificates and so on. Even then, you won't be 100% safe, but you'll be taking an active part in your own mitigation strategy.