Does there exist a common site to test AI of board games against other AIs?

Question

I am wondering if there exists an site where people can upload their AIs to contest against each other in different board games: Chess, Gomoku, etc.

The site would accept source code of programs (written in some common language), compile it and run the programs against each other. All the programs would have to use some common communication technique.

My motivation is that I have seen many different Gomoku programs in Stack Overflow, and I would like to test the different algorithms against each other. But each one uses different languages and interfaces and I have no way to put them to play against each other.

Common dedicated server, that would play the AIs against each other and keep a global score-board would be tons of fun :)

Does such server exist?

The best I could find is http://wawrzak.com/megagomoku/, but it is still something that I have to download and run on my own computer - I would prefer an existing site where anyone can contribute.

EDIT: Also interesting is http://gomocup.wz.cz/gomoku/download.php . It is gomoku contest held each year, and features a common interface for communication and lot of existing gomoku programs. I wish it were ran more often than once in a year, though :) The immediate feedback of uploading Your program and seeing the results would be very good.

Answer 1

There is a yearly competition for things like this! This field of AI is called general game playing and is a relatively new area of study started recently by Prof. Michael Geneserth of Stanford University. Each year at AAAI there is a competition to determine the best GGP program after a variety of games have been played.

Outside of the competition, there are several live servers where you can play against the research universities and amateur hobbyists. The website ggp.org is relatively new but open and distributed, while The Technical University of Dresden maintains a more active server.

To play here you will need to build a player that conforms to the GGP standard protocol. For help in getting started, there is a project called ggp-base maintained by the current GGP world champion with a simple infrastructure for creating GGP players. It might be a great place to start.

Hope this helps!

Answer 2

A common server with a mostly common interface (aside from necessary game-specific differences) would be nice. Uploading your source code and having the server compile and run it has the nice effect of completely eliminating cheating (entering human moves as if a bot came up with them). But it is only practical for very low time-limit games because of the high CPU requirements -- each game engine will pin the CPU for the majority of it's allotted time. The lower the time limits, the more games you can run per day per CPU core.

But still, I like the idea. Even with low time limits, it would be fun. Hmm, maybe I'll start this project...