93

Say, I have on my server a page or folder which I want to be secret.

example.com/fdsafdsafdsfdsfdsafdrewrew.html

or

 example.com/fdsafdsafdsfdsfdsafdrewrewaa34532543432/admin/index.html

If the secret part of the path is quite long, can I assume that it's safe to have such a secret page or area, and it'll be hard to guess or brute force it?

What are the issues with this approach in general?

Note that I'm not asking how to do this right, but what could the issues with this approach be, if any.

marked as duplicate by Tom K., Stephane, Rory Alsop Jul 12 '18 at 5:07

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • 43
    oh, and you've turned off directory listings and "smart error" pages that suggest paths... – dandavis Jun 19 '18 at 6:30
  • 62
    Note that this practice is considered "good enough" by some well-established web services such as Google Docs (which has a share-by-link feature) and Overleaf (a collaborative online Latex editor). – Federico Poloni Jun 19 '18 at 7:50
  • 9
    @FedericoPoloni: Google is likely going to track bulk requests and eventually block them. Without that in place, it becomes more likely to succeed by bruteforcing (relative to the url length of course). – Flater Jun 19 '18 at 7:57
  • 3
    @BgrWorker Tokens in password reset links should be used once only and discarded after that. What OP is asking is a fixed value that can be reused, it's not exactly the same thing – Mr. E Jun 19 '18 at 16:28
  • 4
    @GoodDeeds Share-by-link is one of the allowed sharing options on Google Docs. You can also decide to share to specific accounts, but that's a different thing. – Federico Poloni Jun 20 '18 at 6:12
122

You are essentially asking if it is safe to pass secret parameters in a GET request. This is actually classified as a vulnerability. It is not feasible to brute force a sufficiently long pseudorandom string, assuming the server simply returns a static 404 response whenever an invalid path is specified, but there are numerous other security issues in practice that make this a dangerous technique:

  • Logging software often logs GET requests in plaintext, but not POST.

  • Using GET makes CSRF trivial* when processing active content.

  • Referer headers may leak secret values to other websites.

  • Browser history will retain secrets passed in GET requests.

Your second example has /admin/ in it. This implies to me that knowledge of this path alone would be sufficient to authenticate to the server in an administrator context. This is very insecure and should not be done anymore than /?password=hunter2, a major web security faux pas.

Instead, secrets should be sent in a POST request. If this is not possible to do or if your threat model is exclusively to prevent brute force of the URL (for example, password reset links which are promptly invalidated after they are used), then it should be safe if done carefully. I am not aware of any side-channel attacks that would provide a method to obtain the string faster than brute force.

* Avoiding parameterized GET requests does not prevent CSRF attacks, which is a common misconception as there are various ways to similarly abuse POST (fake forms, etc), but passing secrets in GET does make CSRF easier.

  • 2
    "CSRF becomes an issue. In fact, using only POST for secrets mitigates CSRF." As explained in your linked OWASP article, this does not actually prevent CSRF. – Alexander O'Mara Jun 19 '18 at 5:30
  • 3
    @Kargari Because the server will not tell you how close you have gotten to guessing the string. If you give it the wrong string, it will return 404 whether or not it's 1 character off or 50 characters off. As such, you would have to exhaustively search through all possible strings until you get exactly right. Anyway, I probably should have said not feasible rather than not possible. Edited it now. – forest Jun 19 '18 at 6:29
  • 9
    timing attacks may be possible though; i doubt webservers and OSs use fixed-time comparisons when checking for the existence of paths! – dn3s Jun 19 '18 at 7:14
  • 4
    @Kargari Brute force is very, very slow. For a 30 character alphanumeric string (i.e. a-z, A-Z, and 0-9), you have 62^30 possible combinations which is far beyond what is possible today. – forest Jun 19 '18 at 9:01
  • 4
    You can mitigate that timing attack by storing a hash of the random string. So the time would tell you how many characters of the hash match, but that's not correlated to how many characters of the original string match. – Barmar Jun 19 '18 at 17:19
34

This is a common approach to share public things restricted to the ones who know the URL. An example is Google Docs:

enter image description here

The second option, "Anyone with the link", creates a link similar to yours. Same for Google Photos, Dropbox, ...

  • The advantage is that the diffusion of the content is somewhat limited.
  • The drawback is that this somewhat depends with whom you share the link, where it is published, etc.

One thing you should consider is the possibility to invalidate it easily (by changing/regenerating the link to your data)

  • 1
    This is also a common mechanism for "unsubscribe" links. – Micah Epps Jun 20 '18 at 18:48
  • 3
    This answer cites Google Docs, but (rightly) mentions that Docs doesn't create the link until the owner decides to share it. That's different than just having a link from the moment the content is created, as in OP's case. I cannot brute force an arbitrary user's private docs, only the ones for which they've intentionally made public links. – Knetic Jun 22 '18 at 5:22
  • I would also add that Google has vast knowledge about virtually everyone (or at least every browser; having or not having a Google account doesn't matter, everyone with a browser has an internal account with Google) and they may utilize it in the background when granting or denying access to a document available via the unique link. Question is whether the OP has similar data and tools at their disposal. – Pavel Jun 22 '18 at 7:17
  • @Pavel: I am not sure I understand. What do you mean by "everyone with a browser has an internal account with Google"? And by they may utilize it in the background when granting or denying access to a document available via the unique link? – WoJ Jun 22 '18 at 7:23
  • 1
    @WoJ Google actively tracks browsers and their users (Analytics, Tag Managers, APIs, Fonts, reCaptcha, 8.8.8.8 etc are used very heavily throughout the web), so that when your browser requests the document via the unique URI, Google already knows more than you'd like about you (of course tracking the life of a unique Docs link is a part of the same story). That makes it easy to challenge or even filter out foul players. – Pavel Jun 22 '18 at 9:44
20

Bad idea. A number of times I have seen a "secret" URL very quickly getting search engine crawler hits, and then discoverable by web search. Once I even saw someone set up a copy of a reputable website in a subfolder of his domain, share it with one person, and soon he was emailed a notice warning him that his domain may have been compromised for phishing purposes.

How did this happen? In the latter case, the web browser had a built in anti-phishing feature which sent visited URLs to fraud detection services. In the other cases, perhaps the browser was collecting browsing data and sending it to a search engine to collect user habits.

If you do this, make sure your robots.txt (or headers/meta tags) is set up to tell search engines not to index the content.

The internet is a wonderful way of bringing the world closer together, but unfortunately it gives everyone potentially permanent access to anything you happen to sneeze out.

  • 31
    engines not to index the content. --> not to index my secret page by adding its url into robots.txt? – Kargari Jun 19 '18 at 15:58
  • 33
    Never put any secrets in robots.txt! The best way to make sure everyone finds out about your secret page is adding it to the robots.txt, because then all that needs to be done to discover the secret is to look at the robots.txt. – Moshe Katz Jun 19 '18 at 20:35
  • 31
    @Kargari There's no reason to put the url of the actual secret page into robots.txt. A robots file is perfectly capable of disallowing entire directory hierarchies. If your secret is in /nocrawl/page/sdghskdfjgneowsvnoiernow.htm then a "Disallow: /nocrawl" directive will apply to it. – jmbpiano Jun 19 '18 at 22:21
  • 6
    @Kargari "/nocrawl" is an example not a prescription. Please read up on how robots.txt files work before attempting to use one. – jmbpiano Jun 20 '18 at 14:03
  • 4
    "the web browser had a built in anti-phishing feature which sent visited URLs to fraud detection services"... wait, what? So that fraud-detection feature can see every Google doc or whatever you ever had "shared by link"? – Mehrdad Jun 20 '18 at 22:41
9

If the secret part of the path is quite long [...] it'll be hard to guess or brute force it?

Yes. An attacker would have to guess the whole thing to discover it. Since there are many possibilities it would take an infeasible amount of time.

What are the issues with this approach in general?

The problem with this is that URLs are not considered secret, so they will be stored in the browser, in logs and by any intermediate proxies. That way, your "secret" URL may be exposed.

Using GET makes CSRF trivial when processing active content.

No. In a CSRF attack, the attacker forges a specific request. When using a secret in the URL, the attacker does not even know the correct URL to forge the request to. This means that CSRF is impossible as long as the URL is secret.

  • 2
    It's not secret if the the page is littered with <a href="/lots-of-secrets"> all over. – forest Jun 19 '18 at 6:58
  • Also, if you end up with hundreds of thousands of valid secret URLs in that namespace, you increased the chance of brute force attacks finding SOMETHING by hundreds of thousands.... – rackandboneman Jun 21 '18 at 22:30
4

As others have stated, this isn't a good idea. Such "secret" links that are used to unsubscribe or similar one-off purposes are typically relatively short-lived, and of no use once they've been used once.

My initial thought when I read this question was that the url would not stay secret for long. I thought that perhaps Chrome (or any other modern web browser) would use the data from the address line to initialize a crawl. It turns out that they don't. However, the researchers discovered that plugins might still trigger a crawl:

The results are pretty simple: Googlebot never came to visit either page in the test.

As it turns out, two people in the test did not actually disable their extensions, and this resulted in visits from Open Site Explorer (someone had the Mozbar installed and enabled) and Yandex (due to a different person, though I’m not clear on what extension they had installed and enabled).

This means that once a link is used, it may be compromised by browser extensions. And then there's no need for brute forcing anything.

What is the purpose behind making these links? What are you, OP, trying to achieve? I'm certain that there are other ways to get there...

2

It'd be hard to guess/bruteforce but other ways to obtain the paths may be possible

For example, the url may be indexed by services such as google, bing, etc. This would make your "secret" url appear when an user searches your page in google. It can be solved configuring the robots.txt file, but remember that indexers may ignore it

Links in the application may redirect to the hidden path

In addition, machines in the same network as the user accessing the "secret" page or the web server can see the url, also every intermediary proxy and the user's ISP (Or VPN if he uses one)

Finally, the url may be persisted in the browser's cache and/or history and in the logs on the webserver and proxies

  • 3
    Never put any secrets in robots.txt! The best way to make sure everyone finds out about your secret page is adding it to the robots.txt, because then all that needs to be done to discover the secret is to look at the robots.txt. – Moshe Katz Jun 19 '18 at 20:36
  • 3
    @MosheKatz I didn't say anything about adding the secret to robots.txt, but configuring it so your pages don't get indexed. Something like Disallow: / – Mr. E Jun 19 '18 at 20:46
  • 7
    @MrE you may know that, but people reading your answer may not understand that. – Moshe Katz Jun 19 '18 at 21:02
2

Can secret GET requests be brute forced?

Yes, they can. As much as any type of request without proper security measures.

If the secret part of the path is quite long, can I assume that it's safe to have such a secret page or area, and it'll be hard to guess or brute force it?

No, it's not safe to have such secret page or area. Yes, it will be hard to guess or brute force it.

What are the issues with this approach in general?

Unlike POST requests, GET requests can be easily found in the web browser history.

0

I think that it will better to implement an automatic or manual (so you'll decide who can access to the page) OTP One Time Password to send via email.

In a manual scenario:
- you receive the request from email@example.com
- you grant access to email@example.com
- the script create the link with the OTP
- the link will be mailed to email@example.com
- when the user email@example.com will visit the page the OPT will be flagged and nobody else can reuse that link

of course this system require a database where store the authorized email, OPT and flag field

Not the answer you're looking for? Browse other questions tagged or ask your own question.