No, knowing the public is not required to crack an RSA private key. Given an encrypted message, the attacker only needs to know something which allows him to distinguish a success in decrypting your message from a failure. This something can be anything, commonly it is some knowledge about the data content as for instance the knowledge of a challenge in its clear form, the presence of certain tags, of the data structure, of data statistics as for instance a presence of large number of correct english words or the presence of a checksum.
While not bad in its essence, your idea of keeping the public key private just reduces your public key cryptography to a situation which would most likely be solvable using symmetric cryptography.
One of the big issue with symmetric cryptography was that the sender and receiver had to find a way to exchange a pair of secret keys securely while there is no secure communication channel already established and no way to check each other identity. The main solution for this was to use out-of-band key exchange.
Asymmetric encryption, to which RSA encryption belongs, solved this by proposing a system where the sender and receiver can publish their public keys in a common directory, those public keys being optionally signed by a third-party entity trusted by both parties. This allows to remove the burden of having to exchange keys using out-of-band methods.
Nevertheless, if you like, you can still use out-of-band methods to exchange the public key. It is not necessary, depending on what you intend to do it may be more complex and harder to maintain over time, it will not increase your security in any noticeable way but it will not decrease your security either.
To go (a bit) in your direction, note that all public keys are necessarily made really public. A good example of this is user's SSH authentication keys. While stored in clear form, the user's public is usually not published (as opposed to the server's public key). However this so as long as there no advantage in doing so. As soon as it becomes more practical to store public keys in a trustable directory (an internal company directory for instance), there is usually no issue in doing so.