Usually, the only time I see viruses happen is when someone mistakenly runs an infected file. But could one also be infected by just visiting a website?
Yes, as @TheProWolfPcGames mentioned this is called a Drive-by download. All browser venders regard the ability for drive-by download as a critical security vulnerability, and when a drive-by download vulnerability is discovered the fix or mitigate quickly.
The Java and Flash browser plugins have historically been the source of the majority of drive-by download vulnerabilities and are being phased out by their vendors (Oracle and Adobe), browser vendors (Firefox and Chrome), and removed from some ad networks (Google AdSense).
My advice would be to remove flash and java plugins from your everyday browser and have a seperate browser that is used only on trusted ad free sites that need flash or java to operate.
Yes. This is known as a Drive-By-Download and works via vulnerabilities in the web browser or in plugins (e.g. Flash or Java). They can also be triggered via Social Engineering - for example, if users are used to clicking through Java warnings they may click to allow the Java to run because "that's what they're used to doing".
If you have an up-to-date browser with up-to-date or disabled plugins, then the risk of being hit by such an attack are very low. Note that these aren't technically classed as a virus, as you say a virus is really an infected file that the user actively chooses to run. They are malware however, probably falling into the "trojan" or "worm" categories.
Here's an example of the code in a vulnerability in Microsoft Internet Explorer that can then be adapted to trigger a Drive-By-Download: