First of all, I feel you are quite a mysterious person because no one could pretend to break 4096-bit RSA in the foreseeable future so I wonder why you need such a very long key.

Coming back to your questions now:

But in which operation is this the case? Generating the keys? Encryption? Decryption? Signing? Verifying? All of them?

RSA keys are used in GnuPG for 2 reasons only:

1. To sign communications
2. To encrypt files.

Can I expect problems with other PGP implementations with 8192-bit RSA keys?

The answer is Yes in case:

1. The destination person won't be to decrypt messages with standard secure memory limits he/she has already set by default unless if he modifies that default setting.
2. The destination person does not have a hardware as good as yours since there is lot hardware not supporting anything bigger than 2048 bits.

First of all, I feel you are quite a mysterious person because no one could pretend to break 4096-bit RSA in the foreseeable future so I wonder why you need such a very long key.

Coming back to your questions now:

But in which operation is this the case? Generating the keys? Encryption? Decryption? Signing? Verifying? All of them?

RSA keys are used in GnuPG for 2 reasons only:

1. To sign communications
2. To encrypt files.

Can I expect problems with other PGP implementations with 8192-bit RSA keys?

The answer is Yes in case:

1. The destination person won't be to decrypt messages with standard secure memory limits he/she has already set by default unless if he modifies that default setting.
2. The destination person does not have a hardware as good as yours since there is lot hardware not supporting anything bigger than 2048 bits (there are even platforms that do not handle the memory locking mechanism used to generate RSA keys)