Information Security Stack Exchange is a question and answer site for information security professionals. It only takes a minute to sign up.
Sign up to join this community
I just accidentally copypasted a wget command into the ebay search box and got the following error:
It happens with wget http://google.com or curl http://google.com, or any other URL...
It does seem to sanitise the input and remove slashes if you just enter a URL but not if you precede it with wget or curl. What could they possibly be doing which causes a wget or curl command to bypass their sanitization and produce a different result?
I assume that ebay.com installed a Web Application Firewall, which recognizes your request as a possible attack. Therefore, your request is cancelled and you receive a HTTP 403 - Access Denied. The mod_security WAF for Apache, nginx and IIS behaves similar: If it is in prevention mode, it will also respond with HTTP 403 by default [1].
Most WAFs have some kind of a rule set. They check whether your request matches one of their rules, maybe with regular expressions. I assume further that one of those rules looks like (wget|curl) (http|https)://.* [2]. The "sanitizing" of double forward slashes in your url happens most likely on the application level. Strings like asdf// will also be shortened to asdf.
[1] https://github.com/SpiderLabs/ModSecurity/wiki/Reference-Manual#secdefaultaction
[2] Skipped escaping of forward slashes for the sake of readability
(http|https).*, matching against https://example.com will wrongly identify the protocol (the first match group) as http because the .* will match s://example.com. I'm sure there are also situations where one regexp takes a lot longer to match than the other.
https?? = (http|https) and https? = (https|http). A very specific (but unlikely) use case is stripping http and https from the beginning of the url with something like ^(http|https) which will always leave the dangling s for https.
E-Bay is a secured service, and you are not passing the security credentials with the curl command. Most likely, you need to pass an "Authentication" header probably with either a base64 encrypted username:password combination or an OAuth bearer token.
"curl http://google.com".
curl http://returns results.curl http://changes your search to justcurl http:and give results andcurl http://ggives me access denied.wget http://google.com, though it got sanitized towget http: google.comin the result (I know Data Saver works as a proxy, but I don't know why it "worked")