Correctness, conference ranking, etc

I missed a few events on the blogosphere, because I've been on the campaign trail (aka interviewing), but also because I'm not such a regular reader these days. There was a bit of follow-up to my now-famous post on correctness and SWAT (including whining in emails to the PC chair, which I find hilarious --- but YMMV).

Daniel Lemire writes a post essentially saying that top-conferences are for trend-followers who can't have an original idea. Commenters pick on the opportunity to congratulate themselves for all papers they couldn't get into top conferences (after all, those had truly original insights).

Yes, we do have a bit of trend-following and bias towards some topics, but I like to think that is an unfortunate inefficiency in the system, not the model itself. In any case, until we get the perfect conference system, accept the following reality: top conferences give your ideas a broader audience, generate more following, and make life easier for you in terms of funding, jobs, etc.

So for those of you who think you are the hot shot of hot shots and the first to walk on water in 2 millenia... (and I truly hope most of you think that, since my readers come from a community doing advanced research)

... after you have managed to solve the most important problem on Earth, please accept another challenge: that of explaining to the rest of us why this was the most important problem on Earth, and why we should accept your papers. It doesn't need to work from the first attempt, but eventually your goal should be to get your research accepted in the canon, and hence to top conferences. (Guys who walk on water tend to get attention eventually...)

I will not deny that this is often a serious challenge; most times I feel I am up against the current at STOC/FOCS. But we are living in times of extraordinary activism in research, and if you're lazy about taking on the dissemination challenge after solving the technical challenge, you will soon find yourself in the middle of a research community that is just a wrong fit for you.


Michael Mitzenmacher wonders about the role of conferences, and what our stance on correctness should be. I reiterate my point the conferences are for dissemination, and for the very related goal of marking how important some research contribution looks in the eyes of the community. Journals are bad at that, unless we are willing to revamp the system:

• they have slow turn-around time, in part for an unavoidable reason (correctness checking)
• the paper is not compared against a sample of many papers, rather it is evaluated by a non-anonymous editor
  
• the paper is seen by 1 editor (from a small fixed set), and 1-2 reviewers. In a conference, we have a permanently changing set of people, 3 of which look at your paper, plus a few external reviewers.
  

Of course, conferences are not perfect, but it seems more productive to fight the problems than to trash the whole system.

Now if conference = badge of quality, what should you do about correctness? I am a big fan of the allowing papers to begin with "This is an extended abstract of a paper that has just been accepted to journal ..."

Of course, this mechanism will not be necessary for most papers, but it fixes a big flaw in the system: how to get dissemination/the quality badge for a result which does not sound 100% plausible, and is not too easily verifiable.