1

Applicants for software engineering jobs sometimes go through coding challenges as part of the interview process. The company may send applicants code and a development task with a deadline, and applicants send back the code with modifications based on the development task.

How do companies presenting these coding challenges securely test the code they receive from applicants? Of course applicants could have the same question, how do they securely work on the code sent to them, but given the applicant reaches out to an established company I imagine the companies are more trustworthy in this situation. What precautions should be taken to safely open, explore, and run the code that someone barely known submits?

A good example of the code in question is an entire Visual Studio solution of ~100,000 lines of C++ code, with the dev task requiring modification or addition of within 100 lines throughout that solution.

Maybe the answer is simply a matter of trust but I wonder if there are technical precautions companies take to ensure no applicants can get malicious code running on the company's computers. I tagged virtualization and sandbox because Virtual Machines seem like a reasonable way to sandbox the code in question, but if testing the code requires the computer doing the tests to have extensive and expensive software installed, is this still the appropriate or only method? Are there any kind of anti-malware scans reliable in this case?

||improve this question
  • They are probably executing such code in virtual environments dedicated for code testing. Such environments might have functional simulations but can be air-gapped from the actual production environment. If an application does end up doing something malicious, the virtual environment could be reverted right back to an earlier snapshot. – whoami Jul 19 '17 at 16:57
  • 1
    given that, companies usually want to asses quality of code as well, there is high chance someone will review code manually. But i think it is often based on trust. Bear in mind, usually when you are write solution for your interview 100k lines, often company knows who are you. and if your code will caused any serious problems, they could take some legal actions agains you. As precautions most likely VM. Spawng another VM with docker requires only admin's good will. – user902383 Jul 19 '17 at 17:07
  • Those both sound like decent answers and I may accept one if you add it as such. – cr0 Jul 19 '17 at 17:12
  • How would code be reviewed manually in a safe way @user902383? Could it be downloaded and the files opened in a text editor that will not compile the code, to safely examine it and compare changes against the trusted original solution? – cr0 Jul 19 '17 at 17:12
  • @cr0 I don't think there is any point to compare code with original solution. you are only reviewing new code, and then you are linking it to original code to test it – user902383 Jul 19 '17 at 19:14
4

To a large degree, this is a problem that solves itself. To understand why, we first must understand the goals of a test like this. More specifically, running the code is not high on the list of priorities. Understanding it, however, is.

The problems that will be addresses by the developer are well understood by interviewers, and the interviewers are going to have some expectation of how the problem will be solved, technically. If it's a problem within an existing code base, the first thing that I'm going to do is diff the initial code base and the developers solution to see every single change they've made, and changes that I'm not expecting are going to stand out, and get extra attention. Even the changes I am expecting are going to get a significant level of scrutiny as I make sure I've understood what the developer has done and assess the quality of the work.

So, aside from the fact that if you wanted to do something malicious you have to hide it in plain sight in a changeset that will be comprehensively code reviewed by people who are both very good at it, and also already know exactly what they're expecting to see, but there is the challenge of opportunity. As an attacker, you would have to gain an interview at a company you want to attack, for a position that allows you to do a coding test, and the coding test has to be of sufficient complexity and allow you sufficiently leeway to both insert and hide your attack. And then they reviewers have to actually choose to compile and run it in order to execute the attack. And the environment it is run in, has to allow for whatever the end-goal of your payload is to be accomplished. Ultimately, this requires a lot of stars to be perfectly aligned, and there's almost certainly an easier, less risky way. It just doesn't make a ton of sense as an attack vector.

||improve this answer
  • Great answer! To confirm: examining the code is safe as long as it is not compiled and ran? I assumed (maybe incorrectly) that simply downloading code and opening it in Visual Studio or a text editor for comparison could put a company at risk. – cr0 Jul 19 '17 at 17:47
  • 1
    It's possible that opening a solution in Visual Studio could cause issues. Opening in a text editor however, that would be virtually impossible. You should indeed examine the solution and project files before opening a project in an IDE like VS, but again, if this is existing code, changes to those files will be highlighted by a diff tool and I would expect them to be examined as a matter of course before trying to load the project in VS. – Xander Jul 19 '17 at 18:00
  • Yes if it's safe to open the files in a text editor, identifying and examining all differences in a text editor and using a basic comparison tool would make sense as a safe first step which also acts like a firewall to block malicious code from getting in a vulnerable environment like an IDE or a compiled program. – cr0 Jul 19 '17 at 18:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.