> Let's suppose I have done the most I can to secure my phone on a software level (e.g. encrypted storage, restricted app permissions ... whatever you consider "maximally secure"). Is physical access still game over?

The correct answer is really "it depends on", but it is **not always game over**. Mobile devices are way more security-wise hardened. Let me calm down discussion a bit, let's consider a basic way data is protected in a modern mobile OS:

1. You set a password
2. Password -> symmetric key 
3. Key -> encrypted storage

Considering *you have done the most you can to secure your phone*, then there are 2 options to access your phone:

1. **brute-force your password** (guessed password means access to your data)
2. **do a factory reset** (you loose data, but data stay confidential)

For instance, let's say, if operating system doesn't save password on disk as a plain text, and your mobile device was rebooted (so keys don't stay in memory): your data is safe. It means that **offline attack** is almost impossible. (I said "almost" but it also means the game is not over yet).

Therefore to be sure about **data confidentiality**, be sure you do your best:

1. Choose a difficult password/PIN
2. Don't forget to lock your device with any kind of credentials

And yes, theoretically there is no absolute security: OS may just not encrypt files, or it may leak encryption keys, even cryptography is based on a relative difficulty to solve a particular mathematical problem, or on difficulty to attack a particular cipher.