Possible to detect packed executable?

Question

A packer is a way of obfuscating an executable program, i.e., transforming so the result is still executable and has the same effect when run, but looks different (so it won't be detected by static anti-virus). Bad guys often use custom packers to obfuscate their malware, to make it less likely that anti-virus will detect the malware or to make it harder for anti-virus vendors to reverse engineer the malware and figure out what it is doing.

Is it possible to detect whether a particular executable has been packed with a custom packer? In other words, given an executable, I'd like to classify it as either "has been packed with a custom packer" or "hasn't". Are existing anti-virus tools good at detecting whether an executable has been packed with a custom packer?

I'm especially curious about the following easier variant of the problem. Suppose I know about a particular custom packer that's being used by bad guys. Is it feasible to recognize executables that have been packed with this packer? In other words, given an executable E and a packer P, I'd like to classify E as either "was packed by P" or "wasn't". Are there known techniques for doing so? How effective are they?

Answer 1

There are many known ways to identify packers. The 'most common' packer UPX and its variations are usually flagged as 'suspicious' by Anti-virus engines due to a signature detection in the EXe.

There's a couple of handy tools called:

• RDG Packer Detector which detects specific packers based on signature checking (presumably the same way AV does it

• PEiD detects most common packers, cryptors and compilers for PE files as well as allowing for disassembly (available to download via softpedia)

• A simple signature DB checker in python for you to play with (not sure where to grab the DB from though (try here?)

There's also a list here of a couple of variations of packers

Once you detect the type of packer, you can use automatic unpacker (if one is already available) or you can start manually unpacking it.

Are existing anti-virus tools good at detecting whether an executable has been packed with a custom packer?

No, not really, they'll be looking (like the above tools) for signatures. If you ever spend time in blackhat forums, people advertise 'FUD Crypters/Packers' which means that they've usually modified the way their packers work so that AV's don't pick up the signatures for a while allowing for malware to slip through.

According to symantect (as quoted in this paper)

Symantec has collected a large number of packers - more than 2000 variants in more than 200 families.

The paper mentioned above also has a lot of information regarding the techniques used, the counter-techniques employed and more which will answer the second part of your question.

Answer 2

There are some scientific papers providing possible techniques to detect packed executables automatically.

E.g. Perdisci, Lanzi and Lee link apply pattern recognition to distinguish packed files from non-packed files. The features they use for pattern recognition give an insight to how it works:

Number of Standard and Non Standard Sections
Packed executables often do not follow standard names.

Number of Executable Only Sections
Packed executables often don't have any executable only sections.

Number of Readable/Writable/Executable Sections
Packed executable needs at least one section that is readable, writeable and executable. The executable section of non-packed files doesn't need to be writeable.

Number of Entries in the IAT
A packed executable has usually less imports.

PE Header, Code, Data, and File Entropy
Encrypted code will look like random.

Lyda and Hamrock describe a technique that only uses entropy. link

Answer 3

It is very easy to identify the exe file is packed or not

1) Open file in PEView and check section name. Many times it shows the section as UPX0,UPX1,MPRESS1.

2) Open file in PEView and go to the text section and check the size of row data and virtual size. If the difference between the two values is very large, then consider the file is packed. Sometimes all content of text sections is zero.

Answer 4

if you want to do this programmatically, you should use this (or similar) way/function to detect the packer/protector or compiler signature in the PE and then compare this signature by a list of signatures witch exist in a data base of known signatures (for example "userdb.txt" of PEiD ).

#include <Windows.h>
#include <iostream>
using namespace std;
IMAGE_DOS_HEADER Dos_Header;
IMAGE_FILE_HEADER Pe_Header;
IMAGE_OPTIONAL_HEADER Opt_Header;
IMAGE_SECTION_HEADER ImgSection;
DWORD sections = 0, Count = 0;
void Get_Sign(LPSTR szFileName, DWORD Lenght);
int main() {
Get_Sign(
  "C:\\Program Files (x86)\\NoVirusThanks\\Smart PC Locker Pro\\NSPL.exe" /* used in this example ! */
  , 399);
system("pause");
return 0;
} /* */
void Get_Sign(LPSTR szFileName, DWORD Lenght)
// Lenght=399 by defult
{
DWORD i;
HANDLE hFile;
unsigned char Buff;
DWORD Signature = 0;
DWORD EP, IVA, RAW, UNL, Offset;
DWORD BytRet;
hFile = CreateFileA((LPCSTR) szFileName, GENERIC_READ, FILE_SHARE_READ,
  NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);
if (hFile == INVALID_HANDLE_VALUE) {
  cout << "Error in opening file\n";
  exit(0);
}
else {
  ReadFile(hFile, &Dos_Header, sizeof(IMAGE_DOS_HEADER), &BytRet, NULL);
  if (Dos_Header.e_magic = IMAGE_DOS_SIGNATURE) {
   SetFilePointer(hFile, Dos_Header.e_lfanew, NULL, 0);
   ReadFile(hFile, &Signature, sizeof(Signature), &BytRet, NULL);
   if (Signature == IMAGE_NT_SIGNATURE) {
    ReadFile(hFile, &Pe_Header, sizeof(Pe_Header), &BytRet, NULL);
    sections = Pe_Header.NumberOfSections;
    if (Pe_Header.SizeOfOptionalHeader > 0) {
     ReadFile(hFile, &Opt_Header, sizeof(Opt_Header),
      &BytRet, NULL);
     EP = Opt_Header.AddressOfEntryPoint;
     for (Count = 1; Count <= sections; Count++) {
      ReadFile(hFile, &ImgSection, sizeof(ImgSection),
       &BytRet, NULL);
      if (EP < ImgSection.VirtualAddress +
       ImgSection.SizeOfRawData)
       break;
     }
     IVA = ImgSection.VirtualAddress;
     RAW = ImgSection.PointerToRawData;
     UNL = Opt_Header.SizeOfUninitializedData;
     Offset = EP - IVA + RAW;
     // get size of  uninitialized data
     // UNL = IVA - UNL;
     // Offset = EP - UNL + RAW;
     // get offset of signatures to read
    }
    for (i = Offset; i <= (Offset + Lenght); i++) {
     SetFilePointer(hFile, i, NULL, 0);
     ReadFile(hFile, &Buff, sizeof(Buff), &BytRet, NULL);
     printf("%X", Buff);
    }
    cout << endl << endl;
   }
  }
}
}

(Excuse me if my English is bad)