A Statistical Perspective on the RoboCup Simulator League: Progress and Prospects

This paper uses statistical analysis to demonstrate the progress to date in answering the RoboCup Synthetic Agent Challenge. We analyze the complete set of log data produced by the simulator tournaments of 1997 and 1998, applying techniques such as principal component analysis to identify precisely what has improved, and what requires further work. Since the code that implements our analysis produces its results in real-time, we propose releasing a proxy server that makes statistical analysis available to RoboCup developers. We believe such a server has a crucial role to play in facilitating and evaluating research on the three specific challenge problems of opponent modeling, teamwork and learning. We also suggest that — if RoboCup is to make the most of the efforts of participating researchers — the time is ripe for the institution of a modular team based on a common model.