If Tor is open source and thus its process of concealing your IP is known, then how does it keep anything secure? Couldn't someone look at the source code and figure out how to reverse the routing process used to hide your IP address? I'm sure this has been thought of, but I'm just curious how Tor works.
No, because knowledge of the method is not enough to break it. You would also need to acquire information (e.g. Decryption keys), which simply cannot be obtained by an attacker.
The attacker can't figure out the source IP, or look at the data. Each Tor node only "knows" the source and destination of a block of data that it is handling. It can't open it to view the contents, and doesn't know where it came from or where it's going.
This is an oversimplification, of course. There are some ways you can use Tor, and still leak personal information:
As with every tool, it's important to use it properly. The "About" pages on the Tor website do actually provide a lot of information about how Tor works, and it's strengths and weaknesses. It's a lot of reading, but very worthwhile.
The simple answer is that it isn't. Anyone can operate a TOR exit node and it can easily be used to MiTM your traffic. TOR isn't meant to secure your transactions, TOR is meant to protect your identity, which is still not always 100% reliable. If your host is compromised (which it is still the same risk as browsing without it, if not more dangerous as ads/sites can be MiTM'd or poisoned to serve malicious content)
The fact that TOR is open source does add a layer of possibility to tear down the walls of the protocol and suite, but if the network is the wild wild west, why bother? I am paranoid, so I don't use TOR, I honestly don't have use for it unless I'm accessing something on an .onion site, which is rare anyway. Your better bet would be to use a VPN from a reliable service, but even then you run into the same issues like the ISP giving you away.
The best option:
Encryption is the short answer. Each TOR client chooses a random path across the network and encrypts the packets with nested encryptions that each node it selects can open.
So for example, say I decided to talk to B then F then Q then A then C, I'd take my information encrypt it so that only C can read it and tell it where the final destination is. Then I'd take that, encrypt it so that only A can read it with information to pass it on to C. Then I'd encrypt that with Q... etc until eventually I have an overall packet that only B can read. Each node can unwrap only their routing information and the package that they need to deliver to the next node.
Provided that the nodes in Tor are mostly behaving, then no log should be kept of where the message came from and where it went to, so it should be very hard to track. There is a threat that if sufficient nodes are compromised by one large bad guy, then the network could be compromised, but practically speaking that has proven fairly difficult. Other things like random delays also make it hard for arrival and departure times to provide useful routing approximation of routing without having a combined knowledge of the majority of the nodes in the series.
I'll take a crack at explaining this without technical jargon.
Lets say you want to send a nasty letter to someone, but you'd rather not deliver the letter in person for fear that they might get angry with you.
You can ask a courier to take the letter from your house, and deliver it to the recipient, right? That works, but has the problem that the courier knows both identities, and an evil courier could snitch on you.
You could use two couriers, and instruct the first to pass it on to the second, and instruct the second to pass it on to the recipient. That almost works, because now you are the only person that knows both endpoints to the conversation.
And just to decrease the chances of getting a bad circuit (when all couriers are evil and working together), ToR uses 3 couriers, so add a '
There is one problem remaining; how do we deliver the instructions to each courier without revealing our identity? If you wrote 'pass to B' on one piece of paper, you can give that to courier
ToR solves this final problem with repeated encryption, encrypting the message and instructions repeatedly, once for each courier. In terms of our analogy, this is what you do;
If you follow this through, you will see that none of the couriers can ever have knowledge of the full circuit, and there is no way for the recipient to find out that you sent the nasty letter.