I am experiencing the joys of being on a Program Committee (in this case SWAT'08).
- famous person A submits a paper claiming an interesting result. The writing is bad.
- famous person B, tricked by yours truly into refereeing the paper, claims said result is wrong.
- my principle for (in)correctness claims is to contact the author directly.
- A responds (retaliates?) with a link to a 40-page full version.
- B reads the full version (I guess it's personal by now), and sends back a detailed meta-argument for why the proof is wrong.
I am told to write a negative comment based on the paper being poorly written, and not mention anything about correctness. But somehow, washing my hands of the paper is not quite my style...
The politically-correct (PC?) alternative is to convince the authors that the result is too hard/long/etc for the conference, and they should submit it to a journal where complete refereeing is feasible
Oddly, this reminds me of freshman year, when I was starting in theory. When I heard of conferences and journals, my natural assumption was that papers would just be published in journals, and the best would be invited to conferences (soon after publication), as a label of quality, and to allow quick dissemination of important ideas. I remember a very confused Alex saying "I really mean, they do it the other way around."
![[home]](http://people.csail.mit.edu/mip/images/home.gif){kind=small}
19 comments:
Why did you agree to be on the
SWAT committee if you think the
conference is worthless? Do you
think that anything other than
STOC/FOCS/SODA is worthless in
the area of theory/algorithms?
I agreed because (1) I did have papers in similar conferences (eg WADS), so it was some kind of duty. (2) I am told it looks good on your CV.
There are good papers everywhere (I've seen people making incredibly stupid calls about where to submit). But below STOC/FOCS/SODA/SoCG/equivalent, the name doesn't mean anything beyond "published somewhere". Of course, a paper "published somewhere" can rise to "a classic" on its own merits.
A conference doesn't have to have the same caliber as FOCS/STOC/SODA/etc. to be considered worthy (e.g. SWAT/WADS)
You are entitled to your opinion, but it certainly wasn't 'humble' as you claim. It was ignorant and arrogant. Your comment is unnecessarily offensive and disrespectful to anyone who submits to these conferences. You really ought to pay more attention to filtering out your ambition-induced half-baked biased perceptions about the world you are privileged to have around you.
A conference doesn't have to have the same caliber as FOCS/STOC/SODA/etc. to be considered worthy (e.g. SWAT/WADS)
You are entitled to your opinion, but it certainly wasn't 'humble' as you claim. It was ignorant and arrogant. Your comment is unnecessarily offensive and disrespectful to anyone who submits to these conferences. You really ought to pay more attention to filtering out your ambition-induced half-baked biased perceptions about the world you are privileged to have around you.
I second that.
You really ought to pay more attention to filtering out your ambition-induced half-baked biased perceptions about the world you are privileged to have around you.
Now aren't we getting poetic?
You are entitled to your opinion, but it certainly wasn't 'humble' as you claim. It was ignorant and arrogant. Your comment is unnecessarily offensive and disrespectful to anyone who submits to these conferences. You really ought to pay more attention to filtering out your ambition-induced half-baked biased perceptions about the world you are privileged to have around you.
I second that.
I third that!
I don't know why, but I'm still amazed by some of the things you say and the attitudes you have.
I fourth that, you need a reality check and a real arrogance/self-centeredness check.
I fully agree with the comments. In many of your posts you come off as someone I would not want in my department, no matter how many papers you publish.
See also here
The moral is: don't piss people off; people have a long memory.
What's wrong with rejecting a paper for being so poorly written that an expert in the area is unable to verify the claims being made?
What do you mean SWAT is worthless? Didn't you see that Vijay and I are the invited speakers? That's got to count for something! (Note: that's tongue-in-cheek, everyone!!!)
While I don't condone the silly "worthless" comment in the post, I think the poor boy has been beaten up enough. (Perhaps I'm just sympathetic after all the dumb things I've said on my and other blogs.)
He does bring up an actual issue in the post, on correctness and conference refereeing. Since over at my blog I'm also ranting about PC issues (at mybiasedcoin.blogspot.com -- self-promoting plug!) I might throw in something about the issue this coming week.
First of all, being in PCs does look good on one's CV, and it is a service to your research community. It is good you accepted, and, from what you write, it sounds like you are doing an excellent job as PC member. Well done!
On the correctness issue you raise, I assume that you are one of at least three PC members handling the paper in question.
Have the other cognizant PC members and their sub-referees, if any, been made aware of the purported correctness problem in the paper raised by your sub-reviewer? Did the author(s) of the paper react to the "detailed meta-argument" as to why the result is supposedly wrong? Has the further opinion of some independent expert been solicited?
Typically, during the PC meetings I have been involved in, if a report claims that a paper contains a serious mistake, then at least another reviewer checked that claim. If a doubt persisted, we contacted the author(s) and asked for full details of the proof, the software, and so on. At the end of the day, if the PC could not convince itself within reasonable doubt that the result was correct, then the paper was not selected for presentation at the conference.
It is the job of the author(s) to present their results and proofs in such a way that they are convincing, readable and clear. So I see nothing particularly wrong with the above-mentioned line of action. If the claimed result turns out to be correct, the author will publish it somewhere else, after having spent more time and care in writing up its proof so that it becomes comprehensible to an expert. If the experts cannot convince themselves that the paper is correct, then I believe that the reports should convey this fact to the author(s). In your specific case, this has implicitly already been done.
You also raise the very topical and interesting issues of correctness, of the publication pattern in CS, and implicitly of how careful conference and journal refereeing can be. These are all issues that deserve blog posts of their own, and I am looking forward to reading Michael's take on them.
Hmmm...
Either you are socially inept to realize how offensive (not to mention childish) your comments are, or you actually know but just dont care. If its the latter, then it actually makes you kind of cool.
Boo hoo. Seems like most of the commenters here are quite jealous of mip's accomplishments...
I'm sure some characters were left out. It should read: "worthwhile but less-than-top conference." RIght? :-)
I really admire your work, but it seems that you have not understood that there is work of value that, for one reason or another, does not make it to the "top" conferences. Indeed, here are some alternative measures for the success of a publication:
* Follow-up work, citations in good publications.
* Real-world importance and impact.
These may be correllated with the acceptance criteria of FOCS/STOC/SODA, but are definitely not identical.
Dear Mihai,
apparently several readers found your statement about SWAT (and implicitly about other conferences) offensive. Unfortunately you didn't give any reason for judging SWAT "worthless". Maybe it would help the discussion (and stop the flame-war) if you could explain yourself?
You also said "But below STOC/FOCS/SODA/SoCG/equivalent, the name doesn't mean anything beyond 'published somewhere'"
I'm curious. Which conferences do you mean by"equivalent"? What about Computational Complexity, STACS, ICALP, MFCS, ISAAC, TAMC, COCOON, FCT, etc. (to name only "general" theory conferences)? Are they all the same quality, and mean only "published somewhere"? What about WSEAS? Does it still mean "published somewhere"?
Then there are many "specialized" conferences, WINE, WWW, ESA, WADS, PODC, DISC, Computational Complexit, and so on... I'm curious about your opinion on these...
Interesting post... couple of things to say here.
"Boo hoo. Seems like most of the commenters here are quite jealous of mip's accomplishments..."
I am sure there are who are jealous of his success, and others who are appreciative of it. And then of course there are fanboys like you. Pointing out the lucrative extend of his offensiveness and arrogance is pretty much independent of feelings towards his accomplishments.
"While I don't condone the silly "worthless" comment in the post, I think the poor boy has been beaten up enough... "
He certainly doesn't think this is the case at all. Basically, when you summarize the two replies by him, you get:
-Refereeing for a worthless conference like SWAT was some sort of duty for him he agreed to, not an opportunity or privilege. And hey! at least now his CV looks a little better.
-If Mihai liked your paper in a specialized conference, there is some indication that you made a stupid call. Next time submit papers he likes to conferences he likes.
-Also almost all of your papers not published in stoc/soda/focs/etc are 'published somewhere' just to beef up your paper count.
-Oh he can take criticism. He just won't do anything with it.
"(Perhaps I'm just sympathetic after all the dumb things I've said on my and other blogs.)"
Bottom line is, this is not a case of saying some dumb things because he took a comment personally or he is an aggressive blogger or he pushes his point too hard. It's not a case or misunderstanding etc. He thinks 1) SWAT is worthless, 2)your SWAT paper is either a crappy paper or its good but you are stupid for submitting it to SWAT/etc, and 3)he has no intention of changing his view and maybe consider it was a silly comment. We are all trolling here.
"Dear Mihai,
apparently several readers found your statement about SWAT (and implicitly about other conferences) offensive. Unfortunately you didn't give any reason for judging SWAT "worthless". Maybe it would help the discussion (and stop the flame-war) if you could explain yourself?"
I am not sure I want to hear his rationale for disrespecting a whole community of people. Also, a flame-war usually has opposing sides. His only objection to the comments seems to be that they are too poetic.
It seems to me that this is a case of a great researcher stuck in community that does not appreciate his work, and forces him to waste his time refereeing worthless papers. On top of that, the community doesn't understand what conferences and journals are for, and famous persons in the community can't even seem to agree on what results are correct and what are not.
This is a sorry state of affairs. I think it's really important to enjoy what you do, and your peers play a huge role in that. If I were you, I would cut my losses and simply find a related community that was more appreciative of my braincells, instead of being stuck in community such as this and hating my life forever. Have you considered a career switch to math?
Been there, done that, ate the T-shirt, wore the hamburger, I laughed, I cried, it was better than Cats.
If the paper is so badly written that you can neither verify correctness nor find a concrete bug, that's more than sufficient grounds for rejection. It's the author's responsibility to convince the reader; if you aren't convinced, reject. As a service to the author, you should perhaps explain why you aren't convinced, but keep in mind that it's not your job to convince the author that he's wrong.
We can't deny somebody the label just for proving something hard.
Of course not, but the difficulty of the result should make the clarity of the writing more important, not less.
Oddly, this reminds me of freshman year, when I was starting in theory.
Yes. Yes, it does.
I came to this discussion a bit late, but--holy shit--Mihai's honesty is refreshing. Thank you.
Post a Comment