I'm doing a "share box" where users can enter a link before posting and the app automatically scraps the URL to get a thumbnail and some other info. My question is: What would be a safe number of HTTP redirects to follow until giving up? The library I'm using has a default limit of 10, but I just realized some sites are doing more than 10 redirects. Take for example this curl command:
This doesn't reach the final content:
curl -L --max-redirs 10 http://www.nytimes.com/2012/03/10/world/americas/bernardo-pazs-inhotim-is-vast-garden-of-art.html?_r=0
Increasing the --max-redirs to 11 works:
curl -L --max-redirs 11 http://www.nytimes.com/2012/03/10/world/americas/bernardo-pazs-inhotim-is-vast-garden-of-art.html?_r=0
Can I just set a limit of 15 or 20 without affecting in a negative way the securty (and performance) of my app?