I would like to get your thoughts/inputs on what approach one can adopt in order to perform penetration testing on a web application which has unique strings appended in every HTTP request. Every response from the server comes with a unique token that is then added (by the client side script) to every request that is made from that page. This means that repeating or fuzzing with burp suite is not an option as the application throws users out with repetitive requests or invalid strings.
Burp supports Macros and Session Handling Rules that are designed to cope with this kind of situation. For example, you can configure a macro that fetches a fresh token, and a rule that updates the current request. This can be used with all Burp tools, including Intruder and Scanner. There's some more information here.
The built-in rules cannot update tokens in the URL. However, the Custom Parameter Handler extension lets you do that.
It can be a little bit fiddly to set this up, but it is possible to configure working session rules for most sites. You also have the option of coding your own extension.
Affiliation: Burp Suite is a commercial product of ours.
My approach to this
Write a quick application to ‘index’ the site, using your preferred development language, and export that to a csv file and then import that into your fuzzer as a set of urls to test.
If you don’t have any development skills then you need to find a third party tool or developer to help.