Is Bluetooth here a good solution?
The use of bluetooth itself as described here does not add any additional vulnerability to your design. The purpose of message encryption is to provide confidentiality to the message irrespective of the channel through which the message is being transmitted. Other security properties though would be dependent on the way you implement the overall solution.
Having said that I disagree that the complicated architecture provides any additional security you anticipate. This also feels very much like re inventing the wheel. I will explain why in a bit.
What are the security properties you need for a secure chat?
- Authentication - Only verified users shall be able to send messages and no adversary shall be able to spoof an existing user.
- Integrity - No adversary shall be able to tamper the message while it is in transit [or storage] such that the final decrypted data at the receiver's end would not be the same as the message sent by the sender.
- Repudiation - There should be a cryptographically absolute way of confirming the identity of a sender even if the sender denies it.
- Confidentiality - No person with access to the channel [Like the IM apps in your case] should be able to retrieve the content.
- Availability - An adversary should not have the ability to disrupt your channel of communication.
- Authorization - If there are restrictions based on user type, a user by themselves should not be able to bypass those restrictions.
Here is how you solve it one by one.
In your example, there is only one authentication factor - access to the network isolated phone with your encryption client. If that device falls into another person's hand they would be able to do the exact same operations phone A-1 is authorised to do. Ideally there should be multiple factors of authentication. [Password protection of phone A-1 can be one of it]. If your client package itself is leaked somehow, one would be able to mimic phone A-1 to communicate to phone A-2. If you want end to end authentication to be possible, phone A-1 should sign the content and the signature should be verified at the phone B-1.
There are two potential man-in-the-middle opportunities in the above architecture [A-2 & B-2]. If you want to protect message integrity end to end between phone A-1 and phone B-1 there should be some sort of HMAC validation or a signature validation at the recipient side. Note that HMAC would require a shared secret between the two communicating parties. And this should be shared using a secure key exchange mechanism. Signature on the other hand requires the receiving party to be aware of the public key of the sender. A secure implementation of this might require reliance on a PKI system.
For the recipient to be able to have non-repudiation, each message should be signed using an asymmetric cryptographic algorithm.
From your question it appears that confidentiality is your key concern. There are two parts to ensuring confidentiality. Defining an encryption scheme or a ciphersuite and exchanging keys securely. Encryption can be symmetric or asymmetric. If it is symmetric the key has to be exchanged between the parties. This could be achieved by either encrypting and sending the key using an asymmetric cryptographic mechanism or by using a key exchange protocol such as Diffie Hellman Key Exchange. There are numerous things to be considered while choosing any of the ciphersuites depending on the weaknesses associated with each.
Your application is using an interference prone channel - bluetooth. Bluetooth jammers could be used by an attacker to limit your availability.There are multiple single point of failures in your architecture, in fact all the nodes are single point of failures.
This is a factor to consider if there are multiple privilege levels.
You are introducing two additional nodes to just eliminate the possibility of a potential compromise of phone A-2 and phone B-2. However this opens up other channels of potential compromise - phone A-1 itself and the channel between phone A-1 and phone A-2.
However, the first idea turns to be naive, because it's really hard to
ensure the cellphone itself not to be hacked, for example, the
The same stands true for phone A-1. Adding that extra node does not provide any exceptional security other than reducing the attack surface of delivering payloads. If the intend was to limit access to clipboard you can use methods like this.