I am in the process of looking for a specific keyword on a particularly large website with lots of pages. I'm thinking I'd have better luck if I have a way to see the links within the website. If I want to see all links under a website, for example: company.com/1, company.com/2, company.com/3, etc. will DirBuster be able to pull this information? The website is static if that matters.
At its core, dirbuster takes in a list of common URLs, and tries them.
Also, while it's trying URLs from its wordlist, it has functionality to parse returned HTML content and add those URLs to the list.
So to your question
I want to see all links under a website. Will DirBuster be able to pull this information?
If the website has HTML links to all its pages, then DirBuster might be able to figure that out, but otherwise dirbuster is only as good as the wordlist that you feed into it.
Considering that the website is implemented as a normal informative "brochure" (you mentioned it is static) all the content should be linked, so what you need to do is to crawl all its content then search. There's numerous tools that can do this for you including
curl. A quick search for crawler or http mirror tools will give you alternatives.
dirbuster (and more modern derivatives) would be useful if you are looking for hidden or unlinked content. Files and directories with predictable names that can be guessed which are not linked anywhere on the website from its index page. It is used to find admin interfaces, configuration files, etc. It's not a crawler.
I think you need httrack + grep, it's easy and fast.
You can download the entire accessible website following links dynamically with httrack (install with
apt install webhttrack on a Debian system, else please refer to httrack download page). You'll have a copy of the website locally. So you can easily search on those local files with
grep -rni 'your search here' from the root directory of the downloaded website.
You can hardly beat grep for quick searches (even though you have forks of grep in order to do so).
If httrack is not enough, you can mix a dirbuster / dirb / gobuster / patator / whatever with a wordlist to try to catch hidden endpoints, then re-run an httrack on them to complete the previously downloaded website and re-grep what you're looking for.