I have a client/server application, I want to log in with Facebook to my app. I can get an access token using Facebook login from the client, then send that to server to authenticate the user. My question is, how do I know that it was my app the one who logged in to Facebook? What if another app that the user logged in to with Facebook did that request?
Facebook connect is pretty much oauth openid connect with a few changes. So in order to understand the security model, it can behelpful to understand how oauth works. There are a lot of resources on that available on the net.
Your question isn't quite specific enough to give you a specific answer. I'll assume you're designing a server application designed to be used in browsers (user agents) and that you don't actually worry about which application did the original login request to authenticate the user, but rather that you want to avoid a scenario where a user which isn't authorized to use your server application can use it because he successfully authenticated himself to facebook earlier on, either directly on fb or using another app that uses facebook connect.
The comments to your original question already provide answers to this question:
When you get an access token from fb connect, you must verify it. There's an endpoint facebook provides in order to do that, so your server application must send a verification request to that endpoint. If you don't verify it, just as you fear, you could end up with an access token that wasn't meant for your app, or for another user, or had expired. If you do verify it, you will get back a json reply from the verification endpoint that contains a user id and an app id. You can compare the app id with your own app id to see if the token is meant for your application. An attacker shouldn't be able to produce a token with your app id (partly because your app id shouldn't be known to anyone but FB and you). But I the main reason why nobody can act as if they were your app is because you must provide the app secret as part of the verification process. Only you and FB know that secret. So FB can fail the verification if app secret and the app id in the token don't belong to the same app (there's actually another step involved before verification - you must first get an app access token, in order to use it with the verification endpoint, and in order to get the app token, you must provide your app id and secret).
From the verification response, you can also see whether the user id is what you expect it to be. One you've verified the token, it's authenticated and you can trust what it says about who and which app the token is for. Until you do verify tokens, they're just claims that anyone could make.