TL;DR: No :)

Apart from some OAuth and Google security this answers your second question as well.

TL;DR: Yes, after some investigation, this seems to be the case.

Secondary devices

Backups

After some investigation, decompiling of the app and running test scenarios, we can conduct the following: The app is using several local (SQLite) databases. Messages are stored in these databases unencrypted, and the databases itself are not encrypted either. You can check this yourself by downloading the .db files from the data/ folder. This is the default storage location for WhatsApp in running mode, and is unencrypted for performance reasons most likely. Normal apps should not be able to access the .db files in the data folder, but there are quite a few adb workarounds.

Database files including settings and messages are backuped in Google Drive (if you choose to do so). The app can request a lock on the folder in Google Drive to prevent users from downloading or accessing the backup. The message databases send to the remote backup location are the msgstore.db.crypt[0-12] files, where the last number denotes the protocol version. The databases are encrypted with a key stored in the data folder. This key is in fact stored on the WhatsApp server.

Second assumption: So the worst part is that if you backup on Google Drive, WhatsApp has theoretically access (?) to your (hopefully encrypted) backup and also access to the en-/decryption key on their servers. Or is there at least a separation between Google Drive not readable by WhatsApp itself? Does somebody has more details?

The moment you install a new device and setup your Google Account, the files can be requested by the app and configured on the local device. This includes the axolotl database containing the identity key which is necessary in order to prove your identity to others. The decryption key is retrieved after the proving ownership of the phone number (username) to the newly installed WhatsApp instance. In theory, WhatsApp should not be able to access those files, but only you and Google. Ofcourse, WhatsApp could download the files and send them to another location. But at some point we need to put trust in the app, especially if it's not opensource.

WhatsApp is making local backups as well, usually twice a day in sdcard/whatsapp. These local backups also contain the identity key and the message storage databases, and are in fact encrypted. Once again the encryption key is stored on the remote server together with your WhatsApp profile. This explains why you can move the entire WhatsApp backup folder from one device to another. Without a verified phone number you cannot read the backup files, or use the identity key, however any rooted device can give easy access to the original, unencrypted, database files.

One last word about end2end encryption protocol: It seems it is useless (not against normal hackers, but I think against US surveillance) when at least one of your friend will do a Google Drive backup of their chat (history of chats are retrievable).

The story does not get any better from this point, as we know that Google and other companies such as Facebook and WhatsApp supply files on request under the FISA act. There is no need to strike at the end-to-end communication as there is already a 'backdoor'. E2E only protects against active adversaries on the communication channel, who do not possess the power to demand backup files from party one and the decryption key from the other.

Window of Attack

Suppose we look at the situation from a non-government or 'normal' attacker with limited resources, then the device is the obvious weakness. Any app with root privileges can also access databases, can copy keys and so forth. The default Android ROMs contains many apps running under the system user, but also the vendor's apps shipped with the ROM (and updates) are protected from user intervention, and thus run as system.

Malicious apps are not without risk either. With the correct permissions they have full control over the sdcard storage, and can access the encrypted backups. When the verification SMS is intercepted at the correct moment (hooking in on the SMS receive call) and the phone number is copied, it should be possible to activate a self-controlled WhatsApp instance and to receive the database decryption key. The attack becomes even more plausible if the adversary has control over mobile communications (which governments often do).

The adb attack is even worse since it doesn't require root permissions. Basically a downgrade attack is possible where an older version of WhatsApp is installed via the bridge interface. This so called legacy WhatsApp can be tricked into a full application backup, resulting in a tar archive. The tarball is pulled to the adb server side and extracted. When properly prepared it would take an USB cable and a matter of seconds.

First assumption: The backup key is saved on WhatsApp servers too. Otherwise a local phone to phone backup would not work?

TL:DR: No :)

The protocol is quite complicated and not limited to WhatsApp but works generally like this; The phone that uses chat app is called the first device. This device is the almighty device storing all sensitive data like private identity keys, connection passwords, local encryption keys, attachment encryption keys etc. All other devices are called second devices, this includes WhatsApp web and desktop clients.

Whenever a new device appears it needs to be authenticated via the first device first. WhatsApp does this using an initial QR-code (other apps use SMS tokens), and then via a set of proofs. Eventually the first device decides if a secondary device is allowed control over the sensitive data. If so, the second device receives these keys on request. With these keys the second device can download the contact file and decrypt it using the requested key. Same works for all media messages.

Whenever you install WhatsApp on a new phone, this entire process will take place and the first device will authorize the new phone and immediately locks out one of the instances. You then have the choice to choose one of the phones. If you choose the new phone, ownership of all secrets will be transferred to the second device, which then becomes the first device.

Apart from some OAuth and Google security this answers your second question as well.

First assumption: The backup key is saved on WhatsApp servers too. Otherwise a local phone to phone backup would not work?

TL;DR: No :)

The protocol is quite complicated and not limited to WhatsApp, but works generally like this; The phone that uses chat app is called the first device. This device is the almighty device storing all sensitive data like private identity keys, connection passwords, local encryption keys, attachment encryption keys etc. All other devices are called second devices, this includes WhatsApp web and desktop clients. These clients are essentially dumb.

Whenever a new device appears it needs to be authenticated via the first device first. WhatsApp does this using an initial QR-code (other apps use SMS tokens), and then via a set of proofs. Eventually the first device decides if a secondary device is allowed control over the sensitive data. If so, the second device receives these keys on request. With these keys the second device can download the contact file and decrypt it using the requested key. Same works for all media messages. A note for WhatsApp specific: these key requests live very short, and thus are secondary clients required to regularly ask the first device for a new key to access the data. This causes the annoying 'Phone not connected' alert in WhatsApp web.

Whenever you install WhatsApp on a new phone, this entire process will take place and the first device will authorize the new phone and immediately locks out one of the two active instances. You then have the choice to choose one of the phones. If you choose the new phone, ownership of all secrets will be transferred to the second device, which then becomes the first device which makes the circle complete.

Apart from some OAuth and Google security this answers your second question as well.