"Physical access = game over" is an over-simplification. It absolutely boils down to the outcome of a threat assessment, or what the vendor needs to protect and to what level. The direct answer to your question is a great big 'it depends'.
Smartphones are no different than other devices to the extent that they are computers running an operating system of some description handling data of some kind that interact with other computers or people in some way via peripherals.
The class and range of attacks a device is susceptible to when physical access is possible is very different to the type of attacks it would be susceptible over a network. Conversely, the impact on the ecosystem is also quite different and it could be as impacting or worse on the network side.
Modern/recent operating systems and smartphone hardware have multiple methodologies that aim to secure user data from attackers, whether by means of physical attacks or otherwise. Even "physical attacks" can vary between occasional access (a few minutes, casual access) to unlimited time and expertise in micro-electronics in a lab (such as forensics investigations). But there are aspects that can defeat some (or all) of these features such as local configuration of the device, having weak passwords, guessable (or no) PIN codes, etc. Online backup services, cloud based accounts (Apple/Google) aid in these vectors since most of the data in a device ends up mirrored on the cloud in some way.
However, not all smartphone hardware is born in the same way and not all operating systems in the field are implemented to the same security strength, so there are attack vectors whereby full access is possible against certain hardware/software combinations provided physical access is possible.
This is a very short summary, there is scope to this matter to write books.
