Logging in to a website, We can use RSA to send password from client to server but what about the reverse?

Question

I'm building an android banking app:
Part I
So while the user is signing in he would need to provide his email and password in a form inside the app. Okay, so we can encrypt the password by using RSA and we would send the encrypted message (Let this message be called E) to the server(In my case a php), that would decrypt the data and compare the hashed password... And the server would know the password is correct.

1) Can't a hacker monitoring the data transferred between the user's phone and a wifi be able to know E? So wouldn't he be able to send E to the server and fake being the user signing in?

Part II
Now the server would reply to client, after verification of password, and send him some private data, for example his balance(or private transactions).
case a: We don't encrypt the private data, the hacker here would obviously be able to see the private data(which are supposed to be private). case b: We encrypt the data, using public secret key encryption, so the private key would be inside the app.

2) But wouldn't the private key in this case(case b) be actually "not private" as a hacker could decrypt my apk file and see the private key? What is the solution?

3) In case b, is the method of encryption(RSA) wrong? Is there a better method?

Answer 1

1. What you a describing is known as a replay attack. If you did just encrypt a message with textbook RSA (without random padding or a nonce incorporated into the message) this would be possible. However, when people use RSA to send data, they usual are referring to SSL/TLS, where RSA is the method used to exchange a symmetric key that is then used for the rest of the communication. If you are using chrome and go to security on a page using https, you can see a message like

The connection to this site is encrypted and authenticated using a strong protocol (TLS 1.2), a strong key exchange (ECDHE_ECDSA with X25519), and a strong cipher (AES_128_GCM).

Which shows that ECDHE (Elliptic Curve Diffie-Hellman Ephemeral) was used instead of RSA as a public key encryption and the symmetric key was AES-128. Included in this protocol are measures to prevent against replay attacks. if you aren't using SSL and want to just use RSA, you would have to add a nonce or padding

2. and 3. as mentioned above, you shouldn't need to put a private key in your app. You can use the public key of the server to communicate and establish and symmetric key. Encrypting whole messages with RSA is considered slow and you really should use an existing library to do SSL/TLS or such. And avoid rolling your own crypto. Whether RSA is safe is another question. It certainly is with high enough key sizes (2048), but other key exchange methods have other advantages (like Diffie-Hellman hiding the key from an outside observer, even if done in plaintext).

Answer 2

Is there a better method?

Yes. Use TLS.

Encryption alone will never get you want you want. Even if you address the issues of interception and replay somehow, you are still in a position where the app can't be certain it is talking to the legitimate server. All the privacy in the world doesn't matter if your private communications are going to an unknown party. This issue is known as "trust."

TLS will give you the ability to authenticate the server (i.e. ensure you are not talking to a hacker that has impersonated the server). TLS leverages a PKI that allows it to authenticate the chain of authority and ensure the receiver of your messages is a trusted entity.

Now, you could try to grow your own PKI, but the fact is that you probably do not have your own certificate authority or a means of getting your certificates distributed to your app in a secure fashion. Only PKIs and public CAs have this ability, so truly TLS is your only feasible option.

Answer 3

First, agree with John Wu. Use TLS first.
Then, secure TLS with client certificate, stored in the mobile app bundle.
Third - I suggest not to rely on client side password. So, you can generate a password on server side, take a hash and then - send this hash that will be encrypted by the client's password with a strong cipher. So, stored on the client side result can not become a hash without client's password. It will be decrypted locally to get the hash. If you add some entropy and change session based data signature, it can guarantee that transmitted data strong protected.
But if you want to be sure of data protection - use different networks to transfer parts of session keys. For instance, sms or audio callback code together with ip transmission.