For example, I can log in to stackexchange websites by logging in to external websites such as OpenID, Yahoo, etc. How does this work? How do the websites (SE and Yahoo!, for example) communicate? How does Yahoo! know that it's really stackexchange? How does stackexchange know that I have successfully logged in to Yahoo! and it's really Yahoo!? Does stackexchange know if I log out to Yahoo! after I log in here?
In your particular example, StackExchange uses OpenID. OpenID supports the idea of identity providers -- Yahoo is an Identity Provider, same as Google, MyOpenID, etc. To log in StackExchange asks you which provider you want to use, and then redirects you to that providers login page.
Once you authenticate against the provider, the provider redirects you back to StackExchange with a token. This token contains your identity information. This is generally called Claims Based Authentication, where a Claim is a piece of information about an identity, e.g. your name, and claims are bundled into a token.
Communication is done through HTTP 302 redirects and HTTP POST's all via the user's browser. Yahoo knows its StackExchange because the Yahoo URL contains a parameter saying it's StackExchange. StackExchange knows the identity token is from Yahoo because it is signed by Yahoo.
And here: http://openidexplained.com/
Regarding the identity token - it can take on several forms. In one particular case (which I know OpenID supports) the token is effectively a set of URL parameters. They provide a number of things, including:
If you change the identifying information, you break the signature and StackExchange will refuse to log you in. You also can't successfully re-create a signature from scratch, as you don't know the shared secret.
Another form of token, one which I'm more familiar with, is a long pseudo-random string. Your browser returns from the identity provider with this in the URL. Then the client site (e.g. StackExchange) checks the token directly against the identity provider; the provider looks in its list of recently issued tokens and, if it finds it, returns the client your uniquely identifying information.
Regarding logout / single-sign-out - no, StackExchange doesn't know if you log out of your OpenID service. Likewise, your OpenID provider isn't notified if you log out of StackExchange.
Some systems (Google mail/docs/calendar is a good example) instead use an approach that logs you out of the site/service you're using and your ID provider service. However, all other services are uneffected. For example: