By granting permission to Everyone, you're effectively removing any concept of access control from that file or folder. The implications of this depend primarily on what else happens on your server.
Presumably if you can carefully control all the ways in and all the ways out, then theoretically you can modulate access in such a way as to make file-level permissions unnecessary. But that's quite a gamble.
In fact, part of what you're relying on is not only your own ability to control access to the server, but also the security correctness of every service and every component and every asset associated with the server, which may not even be something you can control. If any component falls to an attack, your permissive permissions may provide the attacker the foothold he needs to leverage access even further and gain greater control over your server.
Removing any layer of security isn't generally considered a good idea. As a rule you want as much security as your usage case will permit, as every little bit helps.