Hot answers tagged web-scanners
No. The number of pages Google (and any major search bot) is willing to crawl on your domain (or indeed whether they are willing to crawl it at all) is based on how relevant they think your domain is. There are plenty of sites with an infinite number of pages. This problem was solved decades ago.
Even without special engineering, certain site architectures can have long recursive paths - certain wikis, for instance. Any decently written bot should be able to cope with such site behaviour, at the very least having a recursion depth limit. I don't think anyone here is going to be able to answer concretely, since none of us have access to google's ...
Only top voted, non community-wiki answers of a minimum length are eligible