Background:
There are a lot of self-hosted web applications these days, and often, more than one for the same purpose. In my case I am looking for a replacement for GitHub or other big tech/cloud git platforms. But I am stack at: security. Once in a while I will look through a program's source code to evaluate a specific concern or understand how something works, but I am not writing these applications, I'm not looking to do a full fledged audit examining all the code or pen-testing using the OSWAP guide. I also have limited time and resources, I cant spend months or years performing full fledged audits of the dozens of choices I may have.
A friend of mine in cyber security encouraged me to use something well known and established.
So I have tried a few "smell tests":
- Does the web application have a posted security policy
- Do the developers discuss security in updates/change-logs/announcements/newsletters
- Is security mentioned as a feature
- Is it buit on reliable technologies, bonus points if its it uses technology that is known to be harden or provide certain guarantees (e.g. the linux kernel using actual linux users or LDAP, the Rust programming language, a garbage collected language like Python, etc.)
- How big is the attack surface, how many dependencies are there, and how secure might they be?
- Do they have any disclosed CVE's and have they fixed them
- How many eyes are likely to be looking at this application, or is there an organization that is monitoring it?
- Related to the previous, how many people/organizations have used this technology, for how long, and do they have significant security needs?
When I think of vulnerabilities in the context of a git server, I think how someone tried to perform a supply chain attack on the PHP interpreter. I also have noticed how much github has stepped up their game (requiring 2FA, getting rid of SSH tokens, a new tier of finely grained to in terms of security) and see how large of a concern security is for gitlab. Not sure why they have done this specifically (I'm thinking, maybe proprietary software for enterprise users), but that makes me think I should take it all the more seriously.
Example 0:
A concrete example is Pagure, I can see its affiliated with fedora, its built on gitolite, which has claims about its usage security (including an "independent audit", which was by a user and was not that detailed, and a claim that kernel.org used it and mentioned it in the FAQ, this may have been the case, but it does not seem to be any longer) though it seems old. There seem to be some notes in pagure's documentation about security but I don't see any sort of concrete security policy.
In my book it seems to fail 1, and 3. I am unsure about 2, 4, and 5. I dont think they have any disclosed CVE's. But its 7. and 8. where I am really unsure if the criteria is met. Is it "big enough" are enough eyes on it?
Example 1:
Another example, source hut: 1 and 2, and 4 pass. Sort of 3 passes. 5 I would say medium. They have a single discolsed CVE which they made posted a detailed resolution for. But I really dont know how many eyes are on it, and my guess would be source hut is the only major organization examining it, and they have a lot of other concerns hosting source hut (which they do with their privately owned equipment).
So I am really unsure about 7 and 8.
Here is my question: How do I know when there are "enough" or the "right" eyes on a web app? How many users leads to how many people spending sufficient time scrutinizing the software? And how many people scrutinizing the software leads to what level of security for a web app? Is this even a good criterion?
Bonus points for the following questions:
- Short of a full fledged audit, how would you evaluate the security of a web app you would consider deploying and potentially entrust data too?
- Is there anything you might add to/change/remove from my "smell tests"?
- (From the main question) How might I "sure up" Criteria 7 and 8 to be less subjective, or might I compare usage/"eyes" on a project required to reach X level of security to other project(s), if so what might they be?
- How much security do might I really need? Supposing I might use this repo website for A: my personal projects B: clients doing freelancing work, etc. in this specific case as an example of applying a policy to select a server software. Do I need to put this level of thought and consideration into it?
