I'm having an issue analyzing the traffic with a client's website. I'm using ZAP proxy to see all request/responses. Now, here's the problem. Say I make a request to: www.example.com/index.html I never ever see that exact request in the proxy. That request triggers a whole lot of other requests for js scripts, and also requests to www.example.com/api/documents, www.example.com/api/users, www.example.com/api/resetpasswords, etc... but never the main request. When i look for the index.html in all the packets, the only match I can see is in the referer field of all the example.com/api requests, each and every one of them have index.html in the referer field.

What kind of redirection is in place here? I tried to wget the main index.html file and its mainly a lot of scripts being called, as well as some comments about angular js. There are no 30X redirects involved, and if there were redirects inside of some javascript, i should see the main index.html request that subsequently loads the javascripts. How is this redirection happening?

  • Clear the cache and try again. Wild idea: their index.html is part of appcache manifest? – Dog eat cat world Aug 31 '16 at 21:18
  • Hi, theres no appcache manifest on this website, but good sugestion! – Percy Sep 1 '16 at 12:53

If the client is a browser, there is not guarantee to what exactly happens. The index could be cached. Also some browsers show redirects in the network console while never send by the server. Start by disabling caches, cookies and other client side storage (or use a private navigation mode). Btw, AngularJS uses extensive caching, so I almost certain its the cache.


There are no 30X redirects involved, and if there were redirects inside of some javascript,

No, there are indeed no redirects involved, but there are a lot of 304 responses. They explain the behavior you ask about, and what happens is the following:

To limit the number of requests, webservers can use a technique called an entity tags. If the webserver supports this and is configured to use them (Microsoft IIS is by default), upon request the server will calculate a hash for this document (based on last time changed) and includes this hash in the response (ETag header). In the next requests the client can send this hash to the server, which matches the hash with the current ETag. If they match, the server can respond with an empty 304 response (not modifier).

This empty response if what you see. Some browsers will not list these responses, because in essense 'nothing has changed'. If you disable the cache, the client will omit the If- headers in the request, causing the server to resend the document.

With caching and ETags:

enter image description here

Without caching an ETags:

enter image description here

| improve this answer | |
  • This is actually a website were i have to log in. If i'm not logged in (after deleting cache, cookies, etc..), everything works as expected, but when logged in the problem persist. I still don't understand how ever, how the redirection works. If the website had a redirect cached, i should still see the original url and the changed url after in the second request, shouldn't I? – Percy Sep 1 '16 at 12:56
  • @Percy Does the site use SSL by any chance? – Yorick de Wid Sep 1 '16 at 12:57
  • Yes, it does, do you think it has something to do with it? – Percy Sep 1 '16 at 13:19
  • @Percy If the site were to use HSTS, some redirects are stored in the browser. Doing so will increase security, but can cause this effect. You might want to run a Qualys SSL scan to verify. – Yorick de Wid Sep 1 '16 at 13:24
  • It uses HSTS, but the url I see on the browser bar is index.html/#something, not /api/something like i see on the proxy. If it were an HSTS redirect I think I should see the /api/something address on the bar with https, it still wouldn't explain this behaviour. I did the Qualys scan and nothing strange showed up. Anyways, the site doesn't return an HSTS header after the request, and the site is not popular enough to be hardcoded in the browsers HSTS list. – Percy Sep 1 '16 at 13:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.