30

I just accidentally copypasted a wget command into the ebay search box and got the following error:

ebay error

It happens with wget http://google.com or curl http://google.com, or any other URL...

It does seem to sanitise the input and remove slashes if you just enter a URL but not if you precede it with wget or curl. What could they possibly be doing which causes a wget or curl command to bypass their sanitization and produce a different result?

  • 19
    "accidentally" ;) – Daniel Feb 15 '17 at 15:50
  • 2
    For all the people accidentally wondering. Accidentally typing everything up to curl http:// returns results. curl http:// changes your search to just curl http: and give results and curl http://g gives me access denied. – MonkeyZeus Feb 15 '17 at 16:50
  • 2
    Kinda reminds me of xkcd.com/1700, though maybe not quite that extreme. – Ajedi32 Feb 15 '17 at 22:03
  • Curiously, with Chrome, when using Google's Data Saver extension, I could bypass wget http://google.com, though it got sanitized to wget http: google.com in the result (I know Data Saver works as a proxy, but I don't know why it "worked") – Andrew T. Feb 16 '17 at 7:58
35

I assume that ebay.com installed a Web Application Firewall, which recognizes your request as a possible attack. Therefore, your request is cancelled and you receive a HTTP 403 - Access Denied. The mod_security WAF for Apache, nginx and IIS behaves similar: If it is in prevention mode, it will also respond with HTTP 403 by default [1].

Most WAFs have some kind of a rule set. They check whether your request matches one of their rules, maybe with regular expressions. I assume further that one of those rules looks like (wget|curl) (http|https)://.* [2]. The "sanitizing" of double forward slashes in your url happens most likely on the application level. Strings like asdf// will also be shortened to asdf.

[1] https://github.com/SpiderLabs/ModSecurity/wiki/Reference-Manual#secdefaultaction

[2] Skipped escaping of forward slashes for the sake of readability

  • 6
    for brevity, (http|https) and https? are equivalent – cat Feb 15 '17 at 13:53
  • 2
    So what are you supposed to do if you're searching for something curly? – JAB Feb 15 '17 at 16:19
  • 1
    @Maurycy Forgive me, I thought I knew my stuff. Which inputs are matched by one of those, but not the other? – cat Feb 16 '17 at 2:13
  • 2
    Yeah. It doesn't make a difference here because the next part of the regex cannot possibly match an 's'. But with something like (http|https).*, matching against https://example.com will wrongly identify the protocol (the first match group) as http because the .* will match s://example.com. I'm sure there are also situations where one regexp takes a lot longer to match than the other. – David Z Feb 16 '17 at 7:40
  • 2
    @cat I usually use regexes for extracting out data and replacing in strings and I actually had this exact thing bite my butt specifically (ie, the order in matching group). In this specific case, https?? = (http|https) and https? = (https|http). A very specific (but unlikely) use case is stripping http and https from the beginning of the url with something like ^(http|https) which will always leave the dangling s for https. – Maurycy Feb 16 '17 at 10:59
-6

E-Bay is a secured service, and you are not passing the security credentials with the curl command. Most likely, you need to pass an "Authentication" header probably with either a base64 encrypted username:password combination or an OAuth bearer token.

  • 12
    The OP isn't actually using curl/wget to request pages from eBay. Instead, they are filling a form on the webpage in their browser with the string "curl http://google.com". – Kevin - Reinstate Monica Feb 15 '17 at 19:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.