Tuesday, August 14, 2007

Informatics Olympiads (III): Understanding Results

Assuming I convinced you that you should pay attention to Informatics Olympiads, here is your practical guide to how to interpret results from the various olympiads out there...

In the sports Olympiad, there are dozens of competitions (gymnastics, lifting, running etc etc), and 3 medals per competition. In science olympiads, there is one competition with more medals. Specifically:

  • the top 1/12 of the competitors get gold
  • the next 2/12 silver
  • the next 3/12 bronze
  • the bottom 6/12 gets nothing (and their names are not public)
Remember that this is among the best students from each country, so even placing in the top half (bronze) is a very significant achievement.

Competitor profiles generally fall in two different categories:
  • Students from very competitive countries (the big Eastern European countries, USA, China, Russia etc) generally have few, but "good" medals. In these countries, the competition to qualify for the IOI is fierce. Simply making it to the top 4 is much more difficult that doing well at the IOI (for reference, 80% of Romanians who qualified got silver or gold). There are many brilliant students who never qualify, and most students qualify only in senior year (after 4 years of training), thus getting a single shot at a medal.

  • People from less competitive countries have more, but usually weaker medals. It is not unusual for the best students from those countries to qualify each year of high school. On the other hand, at the IOI they are fighting an up-hill battle, as they don't have access to the top-notch preparation and months-long training camps that the competitive countries provide.
To give a perninent example for the need to adjust results by country... Romanian students spend up to 2 months on training camps each year, taking an IOI-like exam every other day. If the students bring back less than 3 gold+silver medals, there is a public outcry about the dimishing quality of education (not that there normally isn't...). Across the border, Moldova had never dreamt of a silver medal before Alex Andoni won 2 of them, and isn't expecting another one soon.

4 comments:

Cosmin Negruseri said...

The romanian training camps declined since you left. Now there are about 2 training camps, 1 week each, in that week there are 3 lecture days and 3 contest days. There is another week when ONI happens, with 2 day contests, one day break and then 2 day contests for the IOI training team selection.

MiP said...

A couple of people asked me why I'm not saying that I got 2nd place at IOI'01. (For the record, I also got 2nd place at CEOI'99.) I am asked whether this is sour grapes over Reid beating me.

It's not! :) And here's how I hope to convince you :)

I do believe that the top 3 positions are important (being among the top 20-odd people and getting gold is fabulous, but being really at the top is a bit more memorable).

But realistically, what more do you learn about me if I start mentioning rankings? If say I only had one gold, it would be important. But this way, it's not really providing meaningful information.

Really, rankings are important, and give us some idea, but exact rankings are essentially useless. Just like in research -- past the first 2-3 papers in STOC/FOCS, it's what you publish, not how many you publish.

Constantin said...

"Moldova had never dreamt of a silver medal before Alex Andoni won 2 of them, and isn't expecting another one soon."

Although I happen to agree with the second part (unfortunately), might I ask whose idea is that? Yours or have you talked to some of our teachers / team leaders?

MiP said...

Constantin, I used to know our teamleaders pretty well when I went to the IOI, and now I obviously know some of the old timers very well (like Alex, Dumi Daniliuc, Dan Strătilă).