I have created a private and public key set and encrypted a message to a friend with their public key. I then turned off my machine, and since it is not in persistence mode, my keys were deleted, but I saved my private and public key on a flash drive. Will my friend still be able to decrypt the message? If not, would he be able to once I boot my machine and add my keys to gpg again?
Yes, he will be able to decrypt your message, but you will not be able to decrypt any of his messages. The public key is used to encrypt a message, and as it is intended to be public, anyone who has it can encrypt a message that only the corresponding private key can decrypt. The private key on the other hand is used to decrypt the message. As long as your friend still has their private key, they will be able to decrypt a message you have created with their public key, regardless of when you created it.
Note that, while you can still encrypt messages to him (as you can always get his public key again), you will not be able to sign your messages. The private key is required to sign an encrypted message.