0

I queries a phrase on Google, and when I clicked on the first result, it asked me to login, and wasn't allowing me to proceed without login.

If it matters, the query that I searched was "how to exit out of microstrategy presentation mode". The first result (now this may vary according to your location or past search results), which starts with "No shortcut key to exit presentation mode..." asks for login.

I was wondering how can Google crawl such pages, which require login.

2

These pages simply don't require a login when Google is crawling them but only if a user with a common browser accesses the pages.

In order to have pay-walled or otherwise protected content included into results of search engines, several sites white-list specific crawlers, for example based on source IP address or user agent. Another way (which might be also employed at the specific site you want to access) is not to white-list specific crawlers but to detect if a real browser is used and then require a login.

0

I think in the current case, as @Steffen said, the website white-listed Google IP or its crawler's user agent.

Google did cached the webpage and you can visit it without signing in.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.