There used to be a "vulnerability" where the image could send a
HTTP 401 Unauthenticated response, which would trigger a login screen for the user. If you set this as forum avatar, it would spawn a login popup for anyone visiting a page where your avatar appears. Lots of people will then attempt to log in with some username and password combination, probably the one for their forum account.
A friend of mine found this a few years ago, but nowadays it doesn't seem to work anymore. At least I couldn't easily reproduce it a few months back. Edit: I was wrong, this attack is still possible! /edit
As for XSS attacks this way, you're safe. The browser will, or should, always interpret this as an image no matter what it contains or what headers it sends. You can customize the image based on the request (serving a small image to SMF prechecking the image so that it doesn't downscale it, then large for everyone else). Or feed the browser lots of gif-data until memory runs out or something. But there are no real big vulnerabilities here that allow Remote Code Execution as far as I know.
What you are only moderately safe for are CSRF-attacks. The image can issue a
HTTP 302 Moved Temporarily response and link to a new location. For example it could link to, I don't remember the specific URL, something like
https://accounts.google.com/logout and log you out of google (this worked a few months ago). Or, slightly more maliciously:
Only GET requests can be done this way as far as I know. Or if the image was originally being loaded as POST request, I suppose it could also redirect the POST, but not change the POST-data. So that's pretty safe.
Last but not least, if the attacker controls URLs of for example forum avatars (such as in SMF forums), it's possible to obtain information from visitors such as their IP address. I wrote a tool a while ago that used the
action=who page of SMF to link IP addresses to usernames. When I started displaying that to users (show "Hello $username with IP: $IP" in the image) all hell broke loose. "How could you possibly know that?!" They were mostly early- to mid-teen techies so they knew what an IP address was, but didn't know that I couldn't hack them with it. It is however considered to be personally identifiable information, at least in the Netherlands, so the admins weren't quite happy about this practice. If you don't display it though, nobody will ever know.
Note: If this post seems hastily written, it is. I'm barely awake too. Perhaps I'll clean it up tomorrow if it's too much storytelling and not naming enough concrete facts and vulnerabilities. Hope this helped anyway!