Sure, if someone were to put up a rogue AP in that situation, your phone would handshake with it. People do that all the time.
However, that doesn't give away your password, because with WPA2-PSK there is a challenge response portion to the handshake using the authentication frame. At that point, the WPA2 PSK or Pre-Shared Key (also known as the PMK/Pairwise Master Key) would need to be brute force attacked. In fact, it is essentially identical to an attacker if they observe your device's handshake with your actual AP as it is to get you to handshake with them; it will still take them the same amount of effort to crack your key or get access to your wireless network.
In a functional WPA2-PSK handshake, the PMK would then be used to derive a PTK (Pairwise Temporal Key), which is what is actually used to encrypt your session.
I noticed that above somebody comment and suggested not to broadcast your SSID, which is typically a setting available on your AP/wireless router. This will not do anything against even the most basic attacker because of a couple of reasons. First, when you tell an AP not to broadcast your SSID, that doesn't mean it stops broadcasting (or "beaconing"). Instead, it just sets the SSID field of the beacon frame to null (\x00), so if you just listen to the airwaves for a couple of seconds or minutes, you will find out that there is an AP in range of your antenna.
Secondly, even if the AP/wireless router was completely turned off, or you somehow were able to get it to stop beaconing (Bad idea by the way...) clients will send out what are called "probe requests." These essentially say "I have an AP saved with SSID abc123, are you out there?" The attacker could then derive an SSID from those probe requests and build a rogue AP with that SSID to get your handshake. The best way to prevent an attacker from handshaking with you would be to either use 802.11w (Which almost nobody does at this point because not all hardware supports it), or to 'forget' or not save previously connected to wireless connections.
In short: Don't worry about this sort of attack vector unless you have a weak password.