There is a fraction of our community who is in love with the idea of publishing in journals, and would like to see our conferences go away or assume a radically different role. In many of the cases, it seems to me that the driving force behind this idea is nothing more than snobbery. Basically, the argument goes like this: (1) I think Mathematicians are cool; (2) Mathematicians publish in journals.
If one is willing to admit that a field can persist in stupidy for a long time due to tradition, then one has to also entertain the possibility that Computer Science is the grown-up in the house, while Mathematics is stuck with remnants of an era when travel was hard, and presenting the graduate students to a welcoming community was not considered important.
But it's not possible to poke fun at journals without the pedantic person in the room jumping up and down that conference publications are not formally verified. This view is, in my opinion, entirely deserving of a counterpoint to Lance's essay entitled "Time for Theoretical Computer Scientists to Stop Believing Fairy Tales".
There are basically two reasons to believe a paper is correct, none of which is that some bored editor used up some brownie points with a friend, who(se student) gave the paper a quick read while watching a World Cup game:
- The authors thought seriously about it and wrote down all the details. Regardless of what you think about journals, this should already be achieved at conference level. Yes, a conference is an announcement -- but I care when you announce "I've done this!" rather than "I'm reasonably sure I can do this." It is beyond my comprehension why conferences do not require full proofs (despite several successful attempts in the past).
- Interested people read it. Yesterday, Timothy Chan sent me a breakthrough paper. Between giving two talks, kayaking on the Charles, and driving back from STOC, I really couldn't read it. But today I read it, and flipped it upside down in my mind until I got it. The value of putting such a paper in a journal? (cdr '(a))
If you tend to write readerless publications, abolishing the journal system might create a distinct feeling of loneliness, as you can't even be sure that you have a few constrained readers. My heart bleeds for you.
But if/when people are interested in your paper, it will be checked for correctness. I've been impressed many times by how well this works, and by the dedication reviewers put in during the short time frame of conference review. Many bugs get caught at the STOC/FOCS level --- by people who care. And if a bug is not caught in time, there's no loss: it will be caught when the paper becomes interesting. (One could certainly imagine improvements to the system, like a requirement that all papers be on arXiv with requests for clarifications/bug reports/discussions posted below the fold. But that's a different crusade.)
To summarize, the journal vs. conference debate is an easy cosmetic change that we can pursue to feel like we're changing something, but it's besides the point if correctness is what we want. We should instead be tackling the real issue: how to increase the quality and readership of our papers. Can we reduce the (perceived) expectation on the number of papers one should publish? Fight the Least Publishable Unit philosophy? Achieve more unity in the field? Reduce the number of conferences, while also allowing smaller results to appear (posters anyone?)...
Funny enough, this is well aligned with another hot goal of the day: increasing conference participation. If papers have more readers and people don't need to travel to the Kingdom of Far Far Away to publish uninteresting papers, there are probably more people showing up at STOC/FOCS/SODA.
![[home]](https://lh3.googleusercontent.com/blogger_img_proxy/AEn0k_taGWyilfFzOGSlx35STbeEOTUqt19hX8BRbt6ghUviroUFwqLT01_Aad8xr9S4-XXqrChUtmIJC_Uasa7rmx7uQPq2x-22lJ1yiZi5w2eY6ZpA=s0-d){kind=small}
Web views/day on average
30 comments:
what was the breakthrough paper?
"One could certainly imagine improvements to the system, like a requirement that all papers be on arXiv with requests for clarifications/bug reports/discussions posted below the fold. But that's a different crusade.)"
I think its a related crusade. Journals routinely publish errata for papers which correct even relatively minor errors (typos,
misprints, etc). Conferences do not.
I liked your points. They are some of real reasons of supporters of a journal version. But lets not forget some of the real reasons people don't like publishing a journal version. CS people are **too lazy** to write those full versions, and **too impatient** to wait a year or so for an answer from the journal. There is also the transfer of copyrights and the paywall of commercial journals.
Lets list what we really want:
1. a system for getting credit for publishing
2. more readable papers
3. some sort of checking/peer review
4. archiving
5. getting feedbacks on papers
6. being read/cited
7. ?
After listing what we want, we can start talking about what is good.
ps: here is a link to Lance's last post on the topic:
http://blog.computationalcomplexity.org/2010/04/what-does-it-meant-to-be-published.html
I agreed but what about developing or 3rd world countries researchers who cannot afford paying for a conference. For those researchers or academics or students is almost impossible to participate in conferences like STOC/FOCS due to the almost non-existence of a budget for basic sciences.
Thank you, thank you for this post! There really should be a concerted effort to integrate the arXiv into CS sociology.
It is clear you are just too lazy to do a careful job with writing a paper properly. Mathematics has been an established field much longer than computer science, and your complaining sounds like sour grapes. The major problem with publishing ONLY in computer science conference proceedings (like you do) is the time frame forces papers to be poorly written, poorly refereed, and more often filled with errors (including serious ones).
Journals routinely publish errata for papers which correct even relatively minor errors (typos,
misprints, etc). Conferences do not.
We're past the stage where an errata needs to be printed in hardcover. If the paper has an arXiv site you can publish an errata and much more. For instance, somebody may ask about a clarification for a lemma that is correct but subtle. The author explains it in a comment, and it stays there for the next lost reader! If I teach a course on the paper I can post a link to my course notes. Etc. There are many things you can have written next to the paper; erratas are one of them.
CS people are too lazy to write those full versions
Such laziness cannot be tolerated, really. It should be fought against from the conference level.
What about developing or 3rd world countries researchers who cannot afford paying for a conference
The download issue is easy to fix. You should put the slides online next to the paper. The talks can (easily) be recorded. And ultimately there is the paper itself :)
The upload issue is more tricky. We could allow posters without live participants (but each case has to be approved by the PC to ensure the system is not abused). Presumably, the reward system in 3rd world countries adjusts to the realities, and the authors can get promoted even without oral presentations.
Mathematics has been an established field much longer than computer science, and your complaining sounds like sour grapes.
In case anybody doubted my initial point :) I actually do not understand Math envy. They study abstract pathologies that nobody outside highly trained mathematicians could give a damn about. They are a severely underfunded field who teach calculus to engineers for a living. We are a field that's changing the world, and we get nicely rewarded for it!
The major problem with publishing ONLY in computer science conference proceedings (like you do) is the time frame forces papers to be poorly written
You have obviously not read my papers. But hey, at least you read my blog.
I do not understand people who want to publish in journals. Already TCS lags behind the state of the art in industry; making the publishing take longer would not help.
How about a blog-like peer-reviewed publishing system?
What was the breakthrough paper about.
What was the breakthrough paper about.
Since it's not mine and it's not publicly available, you'll have to wait.
The issue is not journals vs. conferences. (If I care about the result, I don't care if you just publish it on your webpage.) The issue, to me, is the general laziness of many in the community to write a full version, along with our community's tacit acceptance of this status quo.
You say "if/when people are interested in your paper, it will be checked for correctness." This may be true, but ignores several problems with relying on such an approach (when it comes at the expense of having a full version). The most pressing one is this: it might be easy for specialists in an area to fill in the gaps in a paper, talk to the author, listen to presentations, etc. and be convinced of correctness. But what happens when a student reads the paper 3 years from now? Or when a non-specialist tries to read the paper? Somehow, science should not just be about results, but also about communicating these results.
Jon, since you have a blog yourself, I will assume you also don't like it when people write a comment on some idee fixe that they have, without reading what was being written or trying to relate to the points being made.
1) I'm not for status quo. I think all conferences should demand a full version, which can be checked or falsified based on the written word. Furthermore, this full version should be dynamic, allowing comments, clarification requests, explanations, etc.
2) Yes, some level of detail can look like overkill to experts but not nearly enough for a new student. I assure you many journal papers are unreadable if they happen to be the first paper in the field you're reading. Perhaps the student should start with an easier paper, read a survey, take a class. Not every paper is meant to be a survey.
3) People should write more survey / expository blog entries / etc.
The most pressing one is this: it might be easy for specialists in an area to fill in the gaps in a paper, talk to the author, listen to presentations, etc. and be convinced of correctness. But what happens when a student reads the paper 3 years from now? Or when a non-specialist tries to read the paper? Somehow, science should not just be about results, but also about communicating these results.
Jon: There is the question of correctness and the question of understandability and they are correlated: if I find it very hard to understand the paper, it is tricky to verify correctness. We need to fix the former but I think there are less extreme solutions to that. I doubt that problem is solved by moving to journals. Try reading a math paper in an area you are not an expert it. The fact is that journal articles are written to be reviewed and judged by 2/3 superspecialized experts, and hence targeted at them. I have little incentive to explain anything in the intro. Our conference papers are usually written for a more wider audience (at least the intro is written for a random committee member) and it is more likely that a nonexpert would learn something from it.
I agree there are cases where conference papers are missing too many details. We should try to find fixes for that without breaking what is good in our community. Journals also have several disadvantages that we all know of: absurdly long turnarounds, skyrocketing costs, no deadlines, less open decisions (1 editor vs. 30 fresh committee members every time you submit). So while they look attractive when looking at one aspect of correctness, they look worse in many others.
It's as if we found that matches create too much wastage of wood, and considered moving to flintstones since they are renewable, when we should really be trying to invent the electric lighter.
is the general laziness of many in the community to write a full version,
This one is easy to explain. If one takes the time and effort to publish a full version rather than people saying "Bravo!, this researcher has duly followed up on all its conference papers" what we get instead is "this guy is trying to double count his work!".
Rather than the community saying "Bravo, you followed up on your conference paper! After 5 years of keeping us in the dark, we finally get to see the details. Thanks." --- I would rather have the community say "What do you mean you want to announce a result for which you haven't written down the proof?"
I would rather have the community say "What do you mean you want to announce a result for which you haven't written down the proof?"
This is anon 12:27. For sure, the conference+journal double publishing is wasteful. As you might know VLDB has already combined the conference and journal versions.
I would rather have the community say "What do you mean you want to announce a result for which you haven't written down the proof?
How do you propose that the community ensures this? The only way I see that this can be enforced is by requiring the final conference versions to have full proofs.
Requiring only the conference submission versions to have full proofs won't work. I have even has a situation where I asked the authors for full proofs six months after the conference (this was for a STOC paper), and was told that they were still working on it. The incident was four years ago, and the proofs have not yet been sent to me.
How do you propose that the community ensures this? The only way I see that this can be enforced is by requiring the final conference versions to have full proofs.
One possibility is to have a public forum (some kind of comments/discussion page for each paper) where you can publicly ask the authors for specific details, or discuss various bugs and fixes in the paper. One would like to make it non-anonymous so as to try to ensure some lower bound on the quality of comments and questions.
It would be great if arXiv adds the following features (or these does not even need to be implemented by arXiv):
1. follow by rss the comments of people you choose on the papers you choose.
2. some kind of tagging system, so for example Mihai would tag the papers he finds interesting and one can easily go and take a look at them.
3. create groups of people who can then go and create their own list of papers they find interesting.
main idea would be to have some known researchers put some structure(s) on the papers, so it would not be a just a flat list of papers.
I would like to see tags like: read, verified, found interesting, ... by people I select on papers.
How do you propose that the community ensures this? The only way I see that this can be enforced is by requiring the final conference versions to have full proofs. Requiring only the conference submission versions to have full proofs won't work.
I assume you meant the other way. Yes, the full version should be available at conference submission. PCs don't have an obligation to check proofs, but they sure have the right to do so if they wish.
The problem with your post is that once you require full proofs and have a system where errata are included below online accessible versions, you already have something much closer to a journal than to a CS conference. If you actually *require* that errata be published (that is don't leave it up to the authors to decide) and also that an Arxiv version is published with full explanations of anything truncated in the conference submission, then you are really almost there.
Currently we have the terrible situation where incorrect or semi-incomprehensible results (due to their compression to fit in a page limit) lie forever in the most prestigious CS proceedings and only a small clique of people "in the know" ever get to learn about the full and correct version. Unsurprisingly, that clique isn't at all bothered by the situation.
It's funny how Mihai's previous blogpost about what I see as a very important result got exactly 0 comments, while this journals/conferences post gets tens of comments. Are blog readers shallow?
Once you require full proofs and have a system where errata are included below online accessible versions, you already have something much closer to a journal than to a CS conference
Great! What's the problem?
For all our complaining, TCS is a field in full bloom, which is doing pretty damn well. Wouldn't it be nice to fix the obvious things in the current system, that try to turn everything upside down and go to a system that we know has a similar number of issues?
Things are being fixed: full proofs were required at some conferences, more referees reject papers for being hard to read, page limits went away this FOCS and last SODA, etc. But there's no consistency yet, and many changes get reverse unilaterally by an apathetic PC.
The SIGACT guys should be trying to maintain these good changes consistently across PCs, instead of chasing wild goose.
Are blog readers shallow?
wrong conclusion. there is nothing to answer to the previous post. people read the post, go back to their offices and meeting rooms and try to apply/improve-upon what Mihai described. The "discussion" about Mihai's result will happen in the form of conference papers over the next year or two.
Are blog readers shallow?
Actually, this post is far more important than the previous one in the same way that a discussion on the electoral system is more relevant than a specific election (see UK 2010).
Pubzone (http://www.pubzone.org/) has initiated the idea of having people put their comments on the paper and also rate them. Although this is still very far from perfect, it would help if the community would help supporting and giving them ideas.
Why are the CS theorists so obsessed with FOCS and STOC? I know a few different sub-communities of CS, and noone else is so obsessed with their top conferences!
How are we obsessed? They are simply the top conferences, so they get mentioned a lot. (Often they're just examples for conferences in general, i.e. I think all recommendations I make for STOC/FOCS should apply very well to SODA...)
As for being obsessed with top conferences, look up SIGCOMM and SIGGRAPH. We are nowhere near that level.
Mihai,
Interesting post on interesting issues. Here are my 3 cents:
- Journals are not just the domain of mathematics. Most sciences and many fields of engineering are journal-oriented. This does not mean we have to use them, of course.
- To followup on anon 2:58 pm: indeed, writing two different versions (conference and journal) of a single paper is somewhat redundant. It is even more redundant to review them, since the work is not reused (a paper is not likely to be reviewed by the same person at different stages). In a sense we are lucky that few papers are submitted to journals, since otherwise we would have been stuck with up to twice the number of papers we are reviewing now.
-There is absolutely no principled reason behind slow journal turnarounds. I have had a pleasure to serve in the editorial board of IEEE Transactions on Signal Processing for a few months now, and it looks like the turnaround time is less than 3 months, on par with most TCS conferences. In fact, I believe many TCS journals have much faster turnarounds these days.
My overall feeling is that the difference between journals and conference proceedings is somewhat of a red herring. With proper fixes (such as demanding full proofs and timely reviews) we can reduce the drawbacks of each to a manageable minimum. The difference then boils down to stability vs change dilemma: journals (with relatively static editorial boards) provide the former, while conferences (with fresh PCs every year) provide the latter.
Traveling to many conferences is extremely bad for the environment. Everyone must agree with that before any other opinions are put forward. And in this line of thought, electronic access to the papers is essential, being that a conference or a journal.
Post a Comment