5

I posted today a pretty cool PoC where I was able to provide real-time 'security vulnerability feedback' to a developer as he is writing code in VisualStudio.

You can see the video at Real-time Vulnerability Creation Feedback inside VisualStudio (with Greens and Reds) where every time the user makes a change to the code there is an auto-compilation (using Roslyn's C# compiler) and a SAST scan (using Cat.NET)

Although this PoC is quite agressive (I do a compilation and scan on every keystoke which is a bit OTT), here is another video that shows a bigger compilation+scan on save: Real-Time C# Solution Compilation and Security Scanning (using Roslyn and Cat.NET)

What is key about this PoC is that is represent the real-time loop (almost a REPL) that we need for (some types) of security issues.

Next we need to think about the best way to present this information to developers. For example I was thinking that we might want to have a number of colours shown depending on the type of issue, how serious it could be, its exploitability, etc...

Another cool idea is to change the cursor's color or its shape (think bigger or smaller), depending on the number of issues currently outstanding :)

4
  • 1
    How about a big red strikethrough on code that is insecure. Mouseover: "Don't even think about compiling this" ;)
    – Steve
    Jun 21, 2012 at 23:24
  • 1
    I would want it to have its own window. I would want this to be perhaps listed in a similar fashion as the "TODO" and/or Bookmarks. I would want an options to highlight and change the visual appearence of the code as purely an option.
    – Ramhound
    Jun 22, 2012 at 12:39
  • Perhaps link to documentation regarding the vulnerability.
    – chao-mu
    Jun 22, 2012 at 14:49
  • Why not just write this into the VM?
    – atdre
    Oct 16, 2012 at 7:26

4 Answers 4

1

This is a very cool idea, but I think that to succeed it is important to integrate this seamlessly with the existing environment.

To that end, I would suggest NOT to come up with some new, unexpected way to present the information, but to use the existing formats and mechanisms.

For example, underline critical and high-risk vulnerabilities with a red squiggly line - just as Visual Studio does for compilation errors by default, and just as almost all programmers expect. You could mark less serious flaws with less scary colors - blue, green, etc.
I would go one step further - and allow this to be configurable, int VS's Fonts and Colors options dialog, to enable programmers to either have it fit in with their specific settings, or even have it stand out. You could have one setting for Critical, another color for High, etc.

Additionally, just as with compilation errors and warnings, there should be a window listing all currently found issues - the richer this window, the better, of course (filtering, sorting, etc).

This would go a great distance in training developers that security flaws, are bugs just like functional bugs - and, in some cases (e.g. Critical/High), should be treated as compilation errors.

Note that VS's builtin Code Analysis works similar to this (except not realtime, for the most part).

1

Developer here - only use the tools & visual features the are provided within the IDE. Do not try to bring in your own flavor as that's just one new thing they need to learn. After all you are just highlighting a problem area

At no point in the IDE should you dictate the correct form of the code, because you do not understand the context the developer is currently working within. This sort of work should occur within a build pipeline for a release.

If you are blocking a project from compiling because of a perceived security issue, then your tool is a problem and should be removed immediately from any development environment.

Sometimes people want to write vulnerable code as a test harness OR on a existing code base there are controls outside the codebase. Not to mention Roslyn Diagnostic Analyzer is terrible in project scope and is limited to a single unit.

0

How about triggering a security compile when the window loses focus or gains focus? Or after 10 seconds of user inaction? Definitely do it when the file is saved, that's a good one that always happens pre-build.

For developer feedback, how about displaying colored skulls and crossbones in the left side of the pane, in the area next to where the debugging breakpoints get displayed. Color them to indicate severity: something like yellow for "must fix", and red for "OMG!!!" Hovering the mouse pointer over them could pop up a tool tip describing the bug.

Maybe set the background to a light red (or colored) gradient, with the intensity centered on the vulnerability. Or use colored bars alongside the scroll bar, like WinDiff's screen for showing you where differences are. Use black, white, or non-existent line segments for no bug regions, and line segments that color match the skulls representing the vulnerabilities.

At the very least make sure it's outputting errors that break the build.

0

You need to tell the developer that no automated system can find every flaw, and the the developer needs to be on their toes and look for flaws on their own. A false sense of security is the absolute |worst| thing you could provide to a developer.

Trust nothing, test everything.

(On a side note, simply highlighting every dangerous function and providing documentation of why it is dangerous could be very useful. Also, you should recognize you will never be able to solve 1/2 of the serious security problems in a real application with this "solution". The lack of an important control cannot be underlined or highlighted, because doesn't exist.)

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .