What are some attacks I can try against this form of authentication? How do I test if they are cryptographically secure as well?
A correctly generated v4 UUID is always cryptographically random, that's in the spec. Verifying this, short of looking at the source code, is difficult. There are tests you can run (e.g. Burp Suite's "sequencer" tool) if you get a large number of samples, but it needs to be a really large number, and tested for a long time, to have particularly good confidence.
There are no attacks that are specifically possible against v4 UUIDs. They're technically slightly lower entropy than the typical opaque random token, which is usually a full 128 bits instead of reserving a few for the version number, but the difference is irrelevant for now.
However, there are attacks you can try / check for against such opaque token systems in general. Injection attacks against the DB that holds the tokens, timing attacks (difficult but in theory possible) if it's an early-exit comparison, session fixation, session management issues (does it expire, can it be revoked if the client doesn't cooperate, etc.), and so on. Opaque tokens should also be hashed in the DB (a fast hash is fine, no need for a password hash or even a salt) so that somebody who gets read access to the session table can't hijack all the sessions.
There's also the attacks you can do against session tokens in cookies, most notably CSRF, but you can also see if the cookie is scoped correctly and has the right security flags and so on.