I have a GNUPG file that i use to store all my passwords locally(Dont know if it is a good idea). The question is, is it safe to push the GNUPG file to github so i can access it from anywhere?
It's encrypted, so in theory it's safe, provided your encryption key/password is of sufficient quality.
But if you're making your database public, that's pretty much an invitation for people to try cracking it. If your key is not as strong as you think, or if you slip up in some other way (falling for a phishing attack for example), then you are at risk. I think posting your database publicly is a bad idea. Just because something shouldn't be at risk if it's in the hands of attackers, doesn't mean you should volunteer to be a target. Just like a website keeps its password hashes private as a first line of defense, your first line of defense should be keeping your password file out of the hands of attackers. They can't even try to attack what they don't have yet.
I suggest using an online storage service that offers private access (such as: Dropbox, Google Drive, OneDrive, etc.) to store your encrypted passwords for convenient multi-device access but still keep them hidden from most attackers.
Off-topic: have you considered a password manager designed for the purpose? If you want a local-only tool, KeePass, PasswordSafe, and other options exist. I think you'll find the convenience increases quite a bit with the same level of security (or better, as you you don't need to worry about temporary files created on decryption).
The pass password manager works in essentially the way you are proposing by putting each password (plus optional additional information) in an individual PGP encrypted file and checking it into git in order to move it around. While it isn't the most popular password manager it is popular enough that if this caused serious problems it would probably be known.
I use pass myself although I use an ssh account (authenticated via gpg-agent and my PGPCard) on a rented personal VM rather than github although all this does is hide the filenames, named after the sites for which they are used, which pass uses and wouldn't really make a difference for your all in one file scenario.