​ Currently I'm on my way to becoming a full time pentester, which means I do a bunch of CTF challenges and try to break labs that have been set up for me.

The only issue I'm currently facing is how to organize my information, I tend to start out with a bunch of raw data ranging anywhere from nmap scans to manual testing through netcat.

Is there a way to neatly organize all this information or is there software available for me to do this? (I'm using kali for all of my testing purposes so if it's included there: bonus points)

  • Kali has tools built in to do this. Have you looked?
    – schroeder
    Sep 4 '17 at 13:12
  • 2
    Google search term: "pentest data management"
    – schroeder
    Sep 4 '17 at 13:13
  • 1
    @schroeder I disagree that this is necessarily a candidate for closure; the question is "Is there a way to neatly organize all this information or is there software available for me to do this?". If you discount the latter half of that sentence then it remains on-topic.
    – Polynomial
    Sep 4 '17 at 13:52
  • @Polynomial I hear you, but the first part is, as you point out, a very large subject with multiple facets. It's either too broad, or a product recommendation, or lacking in research (there is a lot written on this subject).
    – schroeder
    Sep 4 '17 at 14:20

What data you keep and how you keep it is usually down to individual preference and the requirements of the organisation you work for. As such I'm not going to directly answer your question with a tool recommendation, but rather give you some tips that are agnostic to the testing approach and OS you use.

  • Most critically, do not overthink your process for organising data! A bloated approach will lead to you becoming lazy over time. You may not think so now, but after a year of continuous testing and reporting you'll get slack with it. Keep it simple and easy so that it doesn't feel like a burden.
  • Make a separate directory for each test. Give it a sensible name. We have assessment numbers, so I use those so that I can easily identify it. Keep all of these test directories under a single parent directory named something like "testdata" - sounds simple but I've seen people putting test data into three separate locations and mixing up results this way.
  • Keep a log of what you do and when you do it. I have a simple script for this - it just appends a line to a log file in the current directory, with a timestamp, so I can just type ptlog started nmap on box and it'll write [2017-09-04 14:07] started nmap on box to the file. You don't need to use the same method, it's just important to keep timestamped logs.
  • Don't just log technical stuff. Log the outcomes of conversations you had with the client, particularly if they involve problems with availability or changes to scope. Make sure you say who you spoke to and when. While it's rare that someone will try to dick you over, good logs are fantastic insurance when shit hits the fan.
  • If you're using Burp, Zaproxy, Maltego, Netsparker, or a similar tool that uses projects or state files, save them early so you don't data if/when the tool crashes. Burp used to be an absolute nightmare for this.
  • When using tools like nmap, dirb, sslscan, ssl-cipher-suite-enum, etc. that default to just writing content out to stdout, make sure you're either using the appropriate output settings to log the results (e.g. -oN, -oX in nmap) or are piping it out to a file. Name the file after the target. If you realise you did something wrong but still sent some packets, or needed to cancel the scan, do not overwrite the original log file when you re-run the command - rename the old log first.
  • If you're copying files from a client system, place them into their own folder and don't mix them up with another system's files. This avoids a lot of confusion later down the line when you're trying to work out why the production environment has a database box that is apparently configured to back up to the UAT or development instance, only to realise later that you mixed up the config files. It also makes you look like an idiot. (I can neither confirm nor deny that this is based on a personal experience)
  • BACK YOUR DATA UP. I cannot stress this enough. If you're doing any kind of long engagement with a client and you lose your entire set of testing data 5 weeks into a 6 week test, you're in serious shit. Chances are you'll need to re-assess a lot of things, which may require change requests that simply cannot be completed in time for the end of the assessment window. That leaves your organisation in a situation where you cannot deliver the assessment which you are contractually obliged to, and it's entirely your fault from a legal standpoint. Your company will probably give you a backup solution and I suggest that you use it frequently.
  • 1
    Side note on the backup, which isn't necessarily related to this question, but important to think about: You turn up on a client's site, far away from your home or office, for a 3 day engagement. You boot your laptop and find that your hard disk is dead or unrecoverable. You locked down your UEFI boot settings for security, so you can't boot from USB or optical, and forgot the BIOS password. No other staff are available to take your place. What do you do? Don't answer this question to me, but think about it when you set up your testing laptop and backup solution. This has happened to me.
    – Polynomial
    Sep 4 '17 at 13:38

I am not sure if I understand exactly what you need, but it seems to me that Kvasir is a tool that fits your needs. Check out the github page for this tool: Kvasir

Not the answer you're looking for? Browse other questions tagged or ask your own question.