As a general rule, things should not be made accessible in ways that were not intended or for reasons that were not intended. The best reason to follow this pattern is because you don't know how it might be abused. Just because you can't think of a vulnerability doesn't mean somebody else can't.
But in this case, I have seen this pattern used against a site owner.
As an example, virtual-hosting operations typically run website content under user isolated users. That is,
foo.com runs as the user
bar.com runs as the user
bar, and neither can access the other's files.
But a secondary route for accessing site content, such as
18.104.22.168/~foo/ might run
foo.com content using some alternate context -- perhaps
www-data, for example. And in that case,
foo.com/filemanager.php could be accessed as
22.214.171.124/~foo/filemanager.php, which gives the visitor access through the site using a different set of permissions than the site owner thought.
There are other potential vulnerabilities as well that I can thing of -- things having to do with configuration differences, path differences, access-control, and other such details. All of this is somewhat installation-dependent, but the potential is there.