How can i find all directories and links under a website? Note that there is no link from home page to all other pages. For example if i have a domain like this users.company.com and each user will have a link like users.company.com/john, users.company.com/peter etc. But i don't know how many users are there what are all the links and i want to brute force to check all links. Is there any tool or website providing this service? If i want to write a python script to do this job, where can i get information to develop this in python?
There is a program called dir-buster developed by the OWASP project which i believe will do exactly what you want, to brute-force file and folder names
To quote the site:
And you should be aware that:
Hope that helps answer your question!
I recommend using skipfish:
The snippet below is from
For additional dictionaries, check out fuzzdb. In particular, the files in:
Another: Better Wordlists for Forced Browsing
Burp Spider, part of the Burp suite of tool has a useful spidering tool for identifying common files and directories of web applications. It's another useful option along with those that have already been suggested and from my experience is fairly simple to use.
You also might consider checking out this question which seem similar to yours:
Hope this helps.