GET and POST are methods used in basic linking, embedding and in submitting forms. These kind of interactions between sites predate the development of the same origin policy. If the same origin policy would be extended to incorporate POST and GET too then lots of sites would probably break.
Therefore any POST and GET which can be created by linking, embedding or forms are excluded from the same origin policy. But GET and POST which use custom headers or similar which can only be created with XHR are not excluded because they never could happen in the old web predating same origin policy. See also Would the CORS specification consider a missing Content-Type header to imply a "simple header"? for more information.
PUT and DELETE could be theoretically used for CSRF...However with the same-origin-policy these requests are not executed.
That's only true by default. If the target provides an appropriate CORS policy then PUT and DELETE could be used for CSRF too.