Results:

This page describes the results of the 1st UAI 2006 inference evaluation. The call for participation was described here.

The results are in, are tallied, and are now included on this web page. The organizers wish to thank all the teams and their members for participating in the evaluation. All teams are to be commended as the success of this evaluation was crucially dependent on each and every one of the teams. All the teams did very well (as the results below show). The organizers also wish to thank the significant help provided by the graduate student assistants mentioned below.

All details about the evaluation are contained in the various bullet points listed below.

1. A Powerpoint presentation that was given at UAI-2006 summarizing the main results of the evaluation (PDF version of the talk is also here).
2. technical report summarizing the main results (a bit more detailed than the powerpoint presentation).
3. ASCII documentation of the XBIF BN network file format.
4. A directory listing of a few example networks, and includes inference scores for both the PE and MPE queries.
5. A directory listing of the PE networks in XBIF format (gzipped) that were used for the UAI06 evaluation
6. A directory listing of the MPE networks in XBIF format (gzipped) that were used for the UAI06 evaluation.
7. An ASCII file that gives the key to each of the the BNs (i.e., either where they came from, how they were modified from where they came from, or how they were created).
8. A few useful scripts that we either collected or that we wrote to deal with BN description files.
9. A directory of timing results for the 5 teams, including the raw timing numbers for each team on each BN (both wall-clock and inference times), for PE and MPE, given in various formats. This directory also includes perl scripts that computed average speedups, scripts that check that the resulting scores are within tolerance (how to run these scripts are briefly described in the PDF writeup mentioned above). The perl script was used to produce the tables that are reported in the power-point presentation. The directory also contains the timing numbers as matrices, and matlab code for the workload-based linear programming evaluation (which showed that each team has at least one workload where they are a winner).
10. A directory containing brief technical descriptions of each team's system, including each team's original description, and a few bits of a number of clarifying emails that were sent after the evaluation.