Xbox Live vs TMG

  • Article

Foreword - Added 2011-03-08

As far as I'm aware, nothing significant has changed since the blog linked here - ISA Server is now TMG, sure, but XBox Live and TMG don't officially support one another. This blog post captures something that seems to work for me, but may not work for you. (If you find a better or more reliable way, I'm all ears).

Again - easiest way to ensure XBox happiness is with a compatible non-Strict-NAT router. I don't have one of them; all I have is an enterprise-security-and-firewalling product as my SOHO router, so I made enterprise-security-and-firewall-ade with it.

Xboooox
(If you just want the "how to set it up" bit without the commentary, skip to the next heading).

Continuing in the tradition of trying to get my game on through ISA Server , I decided to try out the Halo Reach beta tonight, and was promptly stumped when I couldn’t access my Account History (I didn’t need to, as it turns out, but I couldn’t, so it was a challenge , so I wasn’t about to let my girlfiend (not a misspelling) watch TV until I’d fixed it).

The Xbox generally worked fine for games, but frequently in the Marketplace, bad stuff would happen (i.e. an error saying something about not being able to access the marketplace now, but sometimes a retry would work, extremely weirdly).

The logs showed that TMG was intercepting the traffic, running it through the Web Proxy Filter, and noticing that it wasn’t (how to put it nicely) valid , so dumping it, with an error message indicating 13 – The Data Is Invalid. (cue indignant hmph )

With the help of Jim Harrison and Bala Natarajan, I ran through some reconfiguration steps; here’s what I ended up with that works:

How I set it up
Toolbox Objects:

Computers:

  • Xbox (just a name for the XBox's IP address – you still have to know the IP for publishing rules (each time; can't just use the computer object), so the Xbox IP should be static/reserved.)

Protocol Definitions:

  • Xbox HTTP - TCP/80 Outbound , not based on HTTP base definition, not bound to the Web Filter . That's important.
  • The next three from before, which seem to work pretty reliably* for online play:
  • Xbox – TCP/3074 Outbound, and 3074 UDP Send and Receive.
  • Xbox TCP Server – TCP/3074 Inbound
  • Xbox UDP Server – UDP 3074 Receive Send

Rules (in this order)

  • 1. Xbox In TCP – Server Publish Xbox IP, using protocol XBOX TCP Server, on External IP
  • 2. Xbox In UDP – Server Publish Xbox IP, using protocol Xbox UDP Server, on External IP
  • 3. Xbox HTTP – Access Rule, Allow only Xbox HTTP, to External, from Xbox IP
  • 4. Xbox Deny Special Rule – Access Rule, Deny only HTTP (that’s normal HTTP , not our special new HTTP), to External, from Xbox IP
  • 5. My general allow/deny rules , including a quite-high-up rule allowing Xbox access to any protocol outbound anywhere (I have that set for all computers, but if you want to be sure, make a special rule just for the Xbox allowing All Outbound to External .) Any {Allow All Outbound} rules must be ordered after that special HTTP Deny rule.

Notes:

  • The reason you need a special Deny rule for regular ol’ HTTP – despite the unbinding of the web filter from the XBox HTTP custom protocol, and being quite specific about the protocol you’re allowing – has to do with the way protocols are collapsed and dealt with by the Firewall engine. For more information, check out Why do I need a deny rule to make an allow rule for a custom protocol work correctly? at the always-amazing Formerly-Known-As-ISA Blog.
  • knowledgeable/nitpicky/interested observers may note that the publishingrules that I have first can actually be pretty much anywhere; I just keep the Xbox rules grouped so they’re all in the one spot. And at the front so they’re processed as quickly as possible; lag bad.
    • Aside: If shifting individual rules a long way up or down, don’t just right-click yourself into RSI - remember you can multi-select rules that are in the way, then right-click and move all of them up or down above or below the rule you’re wanting to shift. It’s not drag-and-drop conweenyent, but then it’s not as susceptible to “Oops I dragged that OU into Domain Controllers” Syndrome either.

Other Settings I’d twiddled but may or may not be relevant:

  • I excluded the Xbox from compression using HTTP Compression exclusions (this shouldn’t be relevant any more with the Deny rule above, as the HTTP filter won’t be inspecting that traffic)
  • I excluded the Xbox from NIS using NIS exceptions (unsure if NIS still fires for tcp/80 when the Web Proxy Filter is out of the way. I guess I could look. Yeah, I’ll do that. After a kill or two. Or eight.)

There. That’s my word count for the month. Ooh, a non-code-locked Blur demo too! I’ve left my console unloved for too long.

More Notes:

  • I've seen mention of other ports being required inbound; I haven't tried them. As far as I can tell, I can do everything through this setup, voice, host games, the works. With the All Outbound allow rule following the special stuff at the top, I haven't experienced a problem (that I know about)
  • The connection test still reports "Strict NAT" as if it's a bad thing. That's OK, I just ignore that.