I have read that the PE loader is responsible for loading executable images from disk. When and where is the control flow exactly transferred to the loader? The PE format is well documented but there seems to be a little info regarding the functioning of the loader itself.
To answer your question, I need to cover the full description of how a new process created.
There's a great description of this in Chapter 5 of Windows Internals 6th Edition Part 1 (ch. 5 being available freely online on the Microsoft website) which explains exactly how all of this works.
I'll paraphrase an overview of what's said in the book, as copying it verbatim would probably have some copyright issues.
The PE loader is exposed by a set of user APIs in
kernel32.dll, under the
CreateProcess family. There are different APIs for doing different things, e.g. running a process under an alternative security context.
Here's how it works:
- The user-mode API validates the input parameters, and converts them to their system (native) counterparts.
- It then opens the executable file and loads it into memory.
- Creates the executive process object in the kernel. This involves populating the
EPROCESSstructure and registering the process in various lists.
- Creates the main thread of the process (stack, execution context, thread object)
- Performs subsystem-specific initialisation - e.g. CLR init for .NET applications.
- Start the main thread (unless it was created with the suspended flag set)
- Load the appropriate DLLs in the context of the process.
Most of this is done at the kernel level, using the appropriate
Ps-prefixed native functions. The full set of steps involved is rather complex (in fact, it takes up 15 full pages in the book) and involves a lot of different actions depending on the susbsystem used.
The tricky part with your question is that the "loader" isn't really something that gets control flow. The instant you call
CreateProcess, you're technically running the loader. However, the kernel part of the loader begins when
ntdll!NtCreateUserProcess transitions into kernel-mode. If we're really strict about it, we might say that the first part of the loader is
PspAllocateProcess, since that's what allocates the initial structures.
ntoskrnl.exe. You could search "wrk" or using IDA to reverse. Contains some PE fields verifying.
These two are most important.
1Could you explain why?– Tom K.Apr 26, 2018 at 12:45
In the book Mastering Malware Analysis: The complete malware analyst's guide to combating malicious software, APT, cybercrime, and IoT attacks [Alexey Kleymenov, Amr Thabet], there're 2 sections in chapter 2 called "Process loading step by step" and "PE file loading step by step" which document how the Windows PE loader is loaded and how it works.
This video LoaderWatch Demo shows you a sample tracing of the Windows PE loader: https://www.youtube.com/watch?v=qxPEV4PJ6XY