Multifactor authentication requires having 2+ of the following:
- Something you know (e.g., secret like a password)
- Something you have (e.g., token or code provided to specified device)
- Something you are (biometrics)
(Also note there are new concepts for multi-factor, but these are the base ones for MFA)
Google is using a system where you have a password and by using a one time code/pin (OTP). This means that someone cannot log into your account without knowing the code, but doesn't mean they can't hijack your session. If they are have compromised your workstation, network, or browser, then they are seeing everything you input, including this pin. This could be via a classic man-in-the-middle or a man-in-the-browser attack.
Security needs to be applied in layers, so the two-step logon provides one component, but its not an absolute assurance or silver bullet. The attacker cannot log in without the code, so if they want to log in, they either need to compromise your phone directly (malware or authorized malicious app which can read text messages), compromise your computer so they can just manipulate the running code or forward the code to an attacker when you type it in, or compromise you through social engineering (e.g., they convince you to forward the code via SMS under some pretext).
So, it makes it a lot harder to compromise your account, but not impossible. i.e. - If your phone is turned off, they would have a hard time. If your phone was off and was not compromised, and you didn't enter it into the computer they would have a hard time. There are some remaining edge cases (e.g., they doing some phreaking to intercept your SMS), but generally its an upgrade in the security of your account.