Web
Faculty of Science UvA
Universiteit van Amsterdam | Intelligent Systems Laboratory Amsterdam
Arnoud Visser's
Activities

Robocup Virtual Rescue - Map Evaluator

Description

Essential for a serious research community, is a number of standardized benchmarks and a way to objective measure progress. For robotic research, the creation of a standardized benchmark is more difficult, because a robot observations are dependent on the actions the robot takes when embodied in the world. The RoboCup Virtual Robot competition is a way to build a standardized world and a standardized challenge (search and rescue) for teams of robot. Progress is measured on three criteria:
  • the area explored
  • the number of victims found
  • the quality of the map
Unfortunatelly, an objective way to measure the quality of a map is not available; until now this is judged in a subjective way. This page describes a Map Evaluator which objectively evaluates the quality of a map, by measuring the usefullness of the map.

Mapping Benchmark

The distribution of sourceforge contains a benchmark for the mapping assignment. This Mapping world contains of two areas, one with easy mobility and one with more challenging mobility.

During the preliminaries of the RoboCup 2008 this map was used as initial test-run. A start position in the easy mobility is chosen. The following selection shows the variety of maps that are produced on a first day of the competition, when teams are still testing their code. One map is small but without errors, another is larger but contain several small orientation errors, the last one is small and has a single but large orientation error. The quality of the maps where not evaluated, and the results not used in the competition.

Map Evaluator

MapEvaluator is a commandline tool with currently four arguments:
  • -m *.tif : occupancy-grid map - the filename of the georeferenced map to be evaluated on usability
  • -v *.txt : victim file - the filename with the positions of the victims, used as target positions
  • -x -1.504 : start position x - the x-position of the CommStation (meters)
  • -y 11.168 : start position y", "the y-position of the CommStation (meters)
This arguments are still under construction, and can be modified in the next release.

Download

When stable, the code will available at the usarsim cvs repository in Tools/MapEvaluator.

Current Version: 0.1

A first Windows binary is available. This binary depends on the Microsoft .Net Framework v2.0 Redistribution Package (x86) and Visual Studio C++ 2005 SP1 redistribution.

Release Note March 31, 2009 This version reads a occupancy grid map and plans a path to several locations (the victims), but has difficulty with free space and always reports that no path can be found.

Links