Is there an attack that somehow floods a target process (e.g. by repeatedly calling a WinAPI function that involves the process) and thereby crashes or temporarily disables it?
Strictly speaking, what you are asking about is a Fork Bomb.
... wherein a process continually replicates itself to deplete available system resources, causing resource starvation and slowing or crashing the system.
In Windows (because you ask about that OS specifically) you cannot spawn new forks at the OS level, but you can create a cascade of new processes. That technique still falls under the "Fork Bomb" term.