Reward Driven Racing in Cambridge

2D AMPS simulatorI just completed my 12 week internship in the Applied Games Group. I was working with Ralf, Thore and Phil on applying model-based reinforcement learning to race car driving. This is one of the most interesting projects I've worked on. Next to my workstation I had an XBox 360 development kit hooked up to a wide screen television, which gave some of the other interns at Microsoft Research the impression that I was just playing video games all day. While I did play quite a bit, the point of my work was to get the cars in Project Gotham Racing 3 to learn to drive themselves through experience.

AMPS at work in PGR 3

I implemented a simplified version of the Adaptive Modeling and Planning System (AMPS) described in my Ph.D. thesis (available here). We found that AMPS learns to drive competently after a single training lap, but the other standard algorithms that I implemented did not perform nearly as well. The strength of AMPS is partly due to its ability to generalize from limited experience through dynamic state abstraction. Currently, most racing games on the market include AI opponents with hard-coded behavior, but I think that opponents that are able to learn and adapt to your driving style and exploit your personal weaknesses can make for some very interesting gameplay.

I would recommend applying for an internship here to any Ph.D. candidate with a passion for games. The projects are fascinating, and the people here are brilliant, coming from all over the world. The atmosphere at Microsoft Research is laid back but intense at the same time, which is exactly what I was looking for in an internship. If anyone has any questions about what it is like to be an intern at MSRC, my current contact information is available on my personal website.

Mykel Kochenderfer


Comments (3)

  1. Anonymous says:

    Cross posted from Ralf, Phil and Thore in the MSR Cambridge Applied…

  2. Anonymous says:

    Ralf, Phil and Thore in the MSR Cambridge Applied Games Group have been continuing their work using F#

  3. shohanali says:

    Freicoin Mafia – Play the game
    Freicoin Mafia is a new RPG game, where you have to fight, to become the most powerful of the mobs. You can make illegal actions like crimes, car theft, blackmailing or you can play gamble games like blackjack, roulette and many more. Follow us on FB for coin
    giveaway code & Redeem cupon.
    Play Game here:>

Skip to main content