.Net dll encryption

Question

By nature, .NET assemblies are extremely easy to reverse engineer. Say I want to hide some code by embedding an encrypted .dll into another .dll that is to be decrypted and loaded at runtime:

1. compile a .dll with all the code I want to protect.
2. encrypt the .dll (with the help of SHA-256 hash algorithm and Microsoft RSACryptoServiceProvider to sign the ciphertext).
3. create a second .dll that will have the encrypted .dll embedded as a resource.
4. Have that second, outer, .dll ask for a key at runtime (from the resource manifest).
5. The outer .dll would then decrypt the encrypted .dll with the key - if signature verification passes.
6. Load and use the .dll.

I want to know whether this method could be subject to preimage attacks and collision attacks or similar attacks on the verification? I am quite new to the world of cryptography, so sorry if this is a dumb question.

Answer 1

It sounds like your second step is actually two steps:

• Encrypt (with some unspecified algorithm) the DLL.
• Sign (with RSA-SHA-256, no RSA key size mentioned) the encrypted DLL.

Step 3: Presumably you are also including the public key (to verify the signature) in the outer DLL?

Step 5:

• Verify the signature on the encrypted DLL blob.
• Decrypt the inner DLL with the key from step 4.

It further sounds like you are most worried about the integrity of the encrypted DLL, for some reason. While I applaud the concern for integrity of the encrypted data, it is entirely misplaced here (I'll explain why in a moment). First, to answer some questions:

• Could this method be subject to collision or preimage attacks on the verification? Technically yes, any hash-based digital signature scheme is at non-zero risk for that. This is why you use a modern secure hash algorithm, such a member of the SHA2 or SHA3 families (both of which have a 256-bit variant). However, even if the attacker has the encryption key, trying to produce a colliding ciphertext (that still decrypts to something meaningful) would be extremely difficult.
• Is SHA-1 good enough for my purpose or not? Plausibly yes in practice, but in theory, no. SHA1 is deprecated and should not be used; there's basically no reason to do so given the newer, much better algorithms.

---

So, why is this whole effort kind of pointless? In a nutshell, it's nigh-impossible to prevent an attacker from introducing malicious code into a process when the process is running on an attacker-controlled computer. A few things an attacker could do:

• Modify the outer DLL to skip the signature check, if for some reason modifying the encrypted DLL was actually needed for any attack and worth the effort.
• Modify the outer DLL to use a different public key for verifying the signature, if modifying the code (rather than data) was too much like work.
• Modify the outer DLL to write the decrypted DLL to disk / send it to the attacker.
• Modify the outer DLL to capture the supplied decryption key and provide that to the attacker for later use.
• Replace the outer DLL with the attacker's spoofed version that performs any of the above, and also does anything else it wants to.
• Drop a modified / spoofed version of any other DLL (including system ones) that the process loads, which can capture the encryption key, read the decrypted DLL after it is loaded, or do anything else it wants.
• Use the debugging APIs (such as CreateRemoteThread) to inject arbitrary code into the process (typically, code that then loads an attacker-crafted DLL, which can do anything).
• Use an actual debugger to do the above.
• Use registry keys and/or environment variables (such as COR_PROFILER) to force an attacker-chosen DLL to be loaded at process creation.
• Run the entire system under kernel debugging, and then suspend the process and examine/modify its RAM to do things like retrieve the decryption key, extract the decrypted DLL, modify the decrypted DLL, modify any other part of the process, inject an attack DLL, etc.

There's also the question of where the decryption key is. If it's in the executable assembly, the attacker just needs to reverse engineer that. If it's stored elsewhere in the installed files, the attacker just has to find it, which is simplified by tracing process execution. If it's supplied by the user, that implies a malicious user has access to it and can use it to decrypt the inner DLL. If it's supplied over a network / the Internet, that implies that the process has some way to authorize itself to the relevant server to retrieve the real key, and the the hunt moves up a lever (now looking for the authorization credential, rather than the key per se).

DRM systems - which it sounds like you're trying to write - evade these issues in various ways (and with various degrees of success), but the really effective ones rely on fetching instructions (and possible executable code) from the Internet, rather than just from the disk, and also constantly cross-checking itself in various ways. In addition, since there's no way to completely prevent access to the process address space and therefore nowhere to truly hide secrets, DRM systems usually have what a former colleague of mine referred to as a "nonsense barrier" - not a security barrier, just stuff aiming to waste a reverse engineer's time and make their head hurt - that is essentially very heavy-duty obfuscation. While it can help with slowing down reverse engineering, this typically comes at the cost of binary size and performance. Most DRM schemes are nonetheless cracked quickly, especially any written by amateurs.

Answer 2

Since you mentioned dll (and not dylib or so), then one important thing to note that, Windows10 would kill your program immediately when it detects some module is applying cryptography on code segments. These are known as "polymorphic virus".

This is because many malwares are using cryptography hide their true functionality. And after many years of analysis, researchers have concluded that the merit of banning code segment encryption outweights allowing it.

Addendum 1

It is mentioned in the comment that applications can be designed to load executable codes over the network. That can be done legitimately, in several ways:

1. Webpages in a browser can load external JavaScripts, and the integrity script element attribute can be used to hint the browser for authenticity verification.

2. Booting over network, requires the computer firmware verify the integrity of the initial bootloader fetched over the network. Likewise, installation of operating systems over the network (like what FreeBSD and some other OSes does) also require some public information about the remote payload to be known by the installer.

Answer 3

Going for obfuscation than encryption is the way to go.

Encryption takes the easy-to-reverse-engineer DLL and adds a layer of encryption, for which one must know the key. Once the key is revealed/leaked, or derived somewhere in the local executable, the original DLL is decrypted and is easy-to-reverse-engineer

Obfuscation takes the source code and scrambles it in a way that it is possible to implement the designed algorithm (the output binary is compliant to the original), but in a way that it is very hard to understand the reversed code.

About DRMs

Digital Rights Management systems are persistently attacked and their inner working is based on the same principles. Without proper hardware encryption, it is not possible to build an indestructible¹ DRM that works on the attacker's device. Even with proper hardware encryption, DRM can be broken

In particular, encrypting a DLL and storing the key (despite hard to find) in the same program is like locking the house with the key and hang it on the door. Remotely fetching the key after authentication is what DRM tries to do, but yet again the key will be somewhere in memory. Only a secure exchange with a hardware token can implement what requested.

Yet, the decrypted DLL, at a certain point, will be stored somewhere in memory. Unless that memory portion is hardware protected even from the OS itself. Chicken and egg?

¹Software encryption is key to failure, but hardware encryption does not mean indestructible or total security

Answer 4

I have responded in-line to quotes from your original question, since some parts of your question seem to me to make sense and some parts of your question seem to me to not be sensible without some assumed context (as evidenced from all the questions in the comments).

By nature, .NET assemblies are extremely easy to reverse engineer.

Yes.

encrypt the .dll (with the help of SHA-256 hash algorithm and Microsoft RSACryptoServiceProvider to sign the ciphertext).

Here, your question starts to become unclear. I think you mean that you will encrypt using some unspecified method and then "sign" with SHA-256. But, SHA-256 is not a "signature" in the usual public-key cryptography sense--sometimes SHA-256 is called a "digital fingerprint," but this is different from a "digital signature," which requires a public and private key (private to sign, public to verify). Or maybe you mean that you will digitally sign the SHA-256 hash using RSA?

Regardless, I can tell you that SHA-256 is resistant to preimage or collision attacks and therefore this type of attack would not be the easiest way for an adversary to attack your hypothetical system.

create a second .dll that will have the encrypted .dll embedded as a resource. Have that second, outer, .dll ask for a key at runtime. The outer .dll would then decrypt the encrypted .dll with the key - if signature verification passes. Load and use the .dll.

There is nothing wrong with this method, per se. However, as others have pointed out, it is certainly possible just to wait until the inner DLL is decrypted and then dump it from memory. It is also possible to obtain the inner DLL by other techniques.

Of course, this does not directly address your question about pre-image or collision resistance, it just provides context. As I mentioned above, you do not have to worry about pre-image or collisions for SHA-256 if you use it correctly, and that is why folks seem to be brainstorming the other relevant attacks for you.

I want to know whether this method could be subject to preimage attacks and collision attacks

The only part of your question that seems relevant to the issue of preimage or collision attacks is the hash function. And your stated hash function, SHA-256, is not vulnerable in this regard in any reasonable period of time.

or similar attacks on the verification?

This is too vague to give a good answer. We can't do a full risk assessment on your hypothetical system since we don't have enough details and some of the details provided don't make sense (or seem to not make sense without some implied context that we would have to guess).

sorry if this is a dumb question.

No problem.

Finally, as others have mentioned, you can use obfuscation techniques to try and slow the reverse engineering process. There are commercial products that are specifically designed to obfuscate .NET assemblies/DLLs and make them very confusing and difficult to reverse engineer. You can use Google to find specific products. If you go this route, I would suggest choosing a reputable commercial product, not freeware.