(tl;dr at bottom)
Encryption is essentially free at this point, even on dial-up connections. Almost every major language has it built-in, or has a library for it, etc. The advantage of not using TLS/SSL is a very small fraction of a second start-up time, a very negligible reduction in CPU usage (a small fraction of a percent), and about 4kb of bandwidth saved per session (TLS has no appreciable effect on bandwidth after the initial handshake). So, assuming you keep the connection open for an entire game, you're saving practically nothing.
However, having the data sent in the clear makes it easy for someone that's interested in playing around with your game to see what it's doing. Once they know what your protocol is, they can then use that to their advantage. A person using a tool designed to do some task is always going to outperform someone that does not have that tool. Just casually thinking, I've come up with botting, high score manipulation, false client updates, tool assisted plays, and session hijacking.
Botting, for example, is having the computer play for you. Computers are really good at looking ahead a million moves in advance, so they can almost always select the correct, infallible move to make in any circumstance faster than a human. This means they'll win, and real players will lose, which ultimately means your game will end up being abandoned or just full of bots playing each other.
High scores mean things to people. It's an ego thing. So, when they play and get a score of 250, and see that the top player has a score of 4,294,967,295, they're not going to bother playing. For example, this might happen if the high score code has an easy-to-break protocol (yes, some games actually just report a high score, and the server just accepts it). That part should be protected from tampering.
False client updates may or may not be a problem, but basically it comes down to if you can "lie" to a client to make it think the game is in a different state than it actually is. For example, people might appear at the wrong place on a map, people can teleport "magically" because the server accepts the coordinates without validating input, etc. You can do this even with encryption, but that means you've had to modify the binary instead of just writing your own client.
Tool assisted plays are similar to botting: the person's still playing, but they're guided by the tool to make optimal decisions, in real time, even if they're losing. A person with a tool will usually perform better than one without (assuming some practice, of course), so this is again usually considered cheating, and impossible to detect short of a watch dog to protect the client. It's even easier if you can modify the packets any way you desire.
Session hijacking could occur if they can figure out the part of the connection that represents their session. You see, you can't just downgrade a TLS connection back to a non-encrypted state (at least, not that I can tell). So, that means you have to drop the connection and open a new one. This new connection needs to identify to the server who it is. Sure, you can use some sort of token, but it'd better be a nonce (a single use value), otherwise the attacker will know the session information, and could possibly trick the legitimate client from continuing, perhaps by replaying the connection to the server, etc.
All in all, you should always use TLS. Even if it's a game of Tic-Tac-Toe or Connect Four. Someone, somewhere, will want to cheat at your game, and you can basically prevent entire classes of cheating simply by enabling a secure channel. Except for a very minor startup cost, encryption is essentially free, so not using it has no real benefits. If you're authenticating securely, it makes no sense to switch to a non-secure channel later, since you've already paid the price of startup.
