Claire Mathieu's SODA and Michael Mitzenmacher's STOC asked for paper abstracts to be submitted one week in advance of the deadline. As the FOCS'09 PC is debating this issue, I wrote the following email expressing my objections. I hope this makes sense to enough people, and we will abandon pre-submission abstracts in theory conferences.
I am strongly opposed to the 1-paragraph pre-submission abstracts, so let me try to argue against them.
Many people, myself included, only write papers in the last week before the deadline. It doesn't matter why this happens, but the PC has to be accommodating. We cannot lecture the community about how they "should" write papers.
( If the argument about being accommodating is not a strong enough for you, I claim that it is, in fact, the optimal rational decision to only write papers the week before. If you're working on several problems at the same time, you're usually stuck waiting for some brilliant idea to move forward. Such ideas come more frequently when you're relaxed and allow your mind to explore improbable paths --- i.e., such ideas tend not to come in the last week. Writing a formal paper, however, is a much more predictable thing: apply brain power, get the proof. This can be done in the last week, even when sleeping little and working with a time constraint. )
If we accept the fact that some authors will write papers in the last week, it means that many of the abstracts will not materialize into papers (if a bug is found, or if the writing is not finished in time --- especially likely if complete proofs are required by submission).
Abstracts that do not materialize into papers are annoying for everybody. For the authors, they are annoying because you have to write an abstract for something that doesn't exist yet, describing some results that might change slightly, and you're not even sure that all proofs will work and you will actually submit!
For the committee, they're annoying because you are reading and bidding on non-existent papers, generating useless work.
I don't see why these disadvantages are worth it. After all, we're only saving a few days (eliminating paper proceedings would probably save a month!).
Cheers,
-mip
Thursday, February 5, 2009
Against Pre-Submission Abstracts
Subscribe to:
Post Comments (Atom)
![[home]](http://people.csail.mit.edu/mip/images/home.gif){kind=small}
20 comments:
The net effect for me of such requirements is to get me to treat the submission deadline as being a week earlier than it was, so that I have a full draft of a paper ready by the abstract submission deadline. This frees me to polish the draft more carefully in the final week, so I think the net is a positive one. The submission deadline is just an arbitrary point in time: why does it matter whether it's a week earlier or a week later?
The deadline is an arbitrary point in time, but your strategy obviously depends on that point in time. What your strategy is doing is spending X-2 weeks thinking about brilliant ideas and 2 weeks writing. I'd rather spend X-1 weeks thinking about brilliant ideas.
Finishing the entire paper by the abstract deadline (i.e. spending just one week of writing) is also suboptimal. You should've thought one week more, and come up with better (or more) results.
What I'm not understanding about your point of view is: if you think your papers require one extra week of thought before you write them, why don't you just start thinking about them one week earlier?
Which is to say, nothing is preventing you from thinking about new research (to be written and submitted in a later cycle) during the week between the abstract submission deadline and the full submission deadline.
if you think your papers require one extra week of thought before you write them, why don't you just start thinking about them one week earlier?
The part where you think about new results is highly unpredictable, and could use as much time as possible.
nothing is preventing you from thinking about new research during the week between the abstract submission deadline and the full submission deadline.
No, writing a paper always takes as much time as you have. There's always something to be clarified, explained better etc.
All arguments about an ideal world are missing the point. People write things up in the last week, and when doing so, writing an abstract is annoying and noisy.
If you are convinced that the PC needs the extra week, just move the deadline earlier.
The real answer is: the whole debate is silly.
If abstracts are required, there is no significant overhead for an author to come up with a title and 1-sentence abstract a week before the deadline. (Worst comes to worst, you can submit a placeholder with the title "On Data Structures" and a vague sentence for the abstract.) The point is to get bidding for papers out of the way early, so even such information can be helpful (and, of course, the vast majority of people will have more informative titles and abstracts).
There is some slight overhead for PC members to have to rank papers that end up not being submitted, and for the PC chair to manually delete papers that have not been submitted. But this overhead is small in the scheme of the 2-3 month review timeframe.
If abstracts are not required, it's not a big deal for people to vote on paper preferences after the deadline. However, I could imagine this causing delay of up to a week which would be relatively high if the review period is short.
In short: it's not a very big deal either way.
On a unrelated note, when should we expect to see the CFP for FOCS 2009?
The real answer is: the whole debate is silly.
Thank you, I couldn't have said it better :) Then, by the general systems principle, among two equivalent solutions you should choose the one with a simpler interface. Not submitting the abstract is the simpler interface.
Bullshit.
Mihai, you're just being lazy. I have absolutely no problem with a system that penalizes people who write their papers at the last minute. Your lack of planning is not the PC's problem. Nobody is obliged to accommodate your lack of planning.
Bullshit, Jeff.
Would you rather have me spend my time on coming up with interesting stuff or on dealing with technicalities to get some papers accepted?
I have trouble understanding your objection as well Mihai. Writing papers does not take up all the available time, otherwise journal papers would never happen. Writing a paper takes as long as it takes to prepare a paper that you and your coauthors are satified with.
Writing papers does not take up all the available time, otherwise journal papers would never happen.
Your argument is irrefutable. I do not have any journal paper outside special issues, which do have deadlines. :)
Fair enough, but what you're doing is taking a personal problem (lack of motivation, outside of conference deadlines, to write down your results) and asking the community to accommodate you.
Note that you yourself point out 2 things that should motivate you to write down your results: 1) you find bugs and 2) by writing things down carefully you often find ways to improve and simplify your ideas. Just write a good paper. What conference you'll submit to can be an afterthought.
Funnily enough, I also don't see much advantage in the abstract pre-submission process. Most of the PC (and their subreferees), although they have 6 (or 7) weeks to review their papers won't start reviewing immediately upon receiving their assigned papers. Most will actually wait until one or two weeks before the deadline. The extra time that the pre-submission buys them is wasted. Of course, I've never been on a PC where there were 50+ papers assigned to each PC member.
You're part of this "crusade of the righteous" that is very annoying in its condescending ways. The motto is something like "We do things the right way, and that fact that you do it differently is a fault of yours."
Why do you think your way is the right one? There are plenty of people writing papers ahead of deadlines (check out the statistics at business meetings, saying that a large percentage of the papers gets submitted the hour before).
Our business in theory is to prove amazing things. Anything else is an artifact, the effects of which should be minimized. If our central goal were to write beautiful papers, we would be doing literature or whatever.
So take pride in your theorems, not in when you write them.
Mihai,
I'm not on a crusade of any kind. Do whatever works for you. I'll do what works for me.
Mihai, I think the point being made is that the overhead of submitting a placeholder title and a 1-line abstract is surely not too much even for you to handle.
But what if someone has two good ideas in the week after the abstract deadline and only submitted one placeholder abstract!
Re: But what if someone has two good ideas in the week after the abstract deadline and only submitted one placeholder abstract!:
Take the time to plan a more carefully written paper for a later submission deadline. Obviously. What are you trying to gain by putting so little time into the writeup? And what are you losing by delaying submission? Sure, you might get scooped, but if it was so easy for other people to find the same idea then it wasn't such a great idea, was it? For that matter even if you have an excellent idea it can probably be made better with a little time for more thought.
It's not as if what you submit must be equal to the final version. You have months to polish the writeup. Furthermore, it's not about what you gain but about what the community gains. The sooner the result is presented, the sooner people can learn about it for the work to have impact (conditioned on you improving the exposition before the final version).
Furthermore, it's not about what you gain but about what the community gains. The sooner the result is presented, the sooner people can learn about it for the work to have impact (conditioned on you improving the exposition before the final version).
Yes, but then I would argue that one should write a polished full version of the paper, post it on one's website, arXiv, CoRR etc., and advertise its existence on the appropriate mailing lists and/or on one's blog. (The paper can then be submitted to the next appropriate conference, and everyone will know who has priority. Moreover, the relevant community will already know the results, making refereeing easier.) This way the community gets to know the results, the proofs and the techniques with the added benefit of a full length and polished presentation. As far as timeliness is concerned, the full paper may very well be accessible before the notification of the accepted papers for the conference whose deadline it missed.
Moreover, one thing that I do not seem to have seen mentioned on this thread of comments is that the devil is often in the details. Ideas that look great when writing proof sketches, sometimes even fairly detailed ones, may end up presenting unforeseen technical difficulties that one cannot overcome. IMHO, this can only be discovered by writing down detailed proofs of results and checking them very carefully.
Re Luca's last point: this is why it's important to take the time to write a full paper with all proofs first, and only then cut it down to the conference submission page limits, rather than writing directly to the conference format. That process doesn't fit very well with the idea of procrastinating on writing until the week before the deadline, of course.
Post a Comment