Posted on

Some “Crazy Ideas” on Computer Science publication process

Scientist investigate to understand natural phenomena, find answers to unsolved questions, in order to ultimately expand the knowledge of mankind. Yes this sounds cliche, but it’s true.

We should remember that science is a social process. It is paramount for scientist to be social and communicate with peers in order to share their ideas, hypothesis, results and receive critical feedback from others. This communication, historically and still today, is done written (papers) and orally (talks and conversations at conferences).

I can only provide my opinion from a computer science point of view, and my impression is that there is a tremendous focus today on reviewing, publications, conferences and citation counts.

I think we are forgetting the big picture: reviewing, publications, conferences are a means to an end. It is the means of communicating in order to achieve the end of understanding something that we do not understand today.

Recently I’ve been seeing a lot of twitter rants and posts about all the problems of reviewing and publications in Computer Science (too many to point to but most notably recently “Effectiveness of Anonymization in Double-Blind Review”   and Michael Stonebraker’s keynote at ICDE 2018  (video). They are all worth reading/listening. However, I think we should think about this situation from scratch.

Some ideas/comments (inspired after a conversation with Wolfgang Lehner when I visited TU Dresden and talking to other colleagues in different sciences)

  • Papers published in top tier conference proceedings have a lot of weight in Computer Science. Fellow colleagues in other scientific fields find it amusing when we get excited about a conference publication because in most (all?) areas of science, journal papers are what counts. I know that the excuse is that CS moves so quickly and journal reviews take too long. So this is what we have to change. Therefore …
  • Conference submissions should be extended abstracts. Reviewing for these extended abstracts should be light. That means that we have less work as reviewers.
  • All accepted abstracts are presented as posters. The goal is to foster discussion, right?!We should have longer poster sessions everyday so everybody can have a chance to present and also see what others are doing.  For example, recently, I received an email from a colleague who sent me a draft paper which was partly inspired by our discussion that we had in SIGMOD 2017. I didn’t even have a poster at the conference, it was just a conversation we had on our way to the gala dinner. We need to have more outlets for these types of conversations.
  • Journal papers get invited for longer presentations. ISWC and WWW (now TheWebConf)  has been doing this for a couple of years now (and I’m sure other conferences too). All conferences should be doing this!
  • During the  conference presentations,  we should spend more time on discussion in addition to giving just talks, and having the session chair asking a question at the end because everybody is looking at their laptop. One idea is to create panels by grouping scientists who are working on the same topic. This is common in social science conferences.
  • We should publish in venues that don’t limit you on a fixed page limit (i.e. journals). Have something to say. Say it. Finish when you are done saying it (and give it a good title… this is advice that I heard from Manuela Veloso.)
  • – Research on a deadline is either engineering or not really research. Therefore, we should not focus on fixed yearly deadlines; you should be able to submit whenever you want. Submit when your work is actually DONE! Not when the conference deadline tells you. That way you can stop running experiments last minute (btw if you do that, your research is not done and not ready to be shared, IMO). I think the PVLDB model is fantastic and should be widely adopted in CS. I know that the typical excuse is that we need deadlines. BS! If you can’t manage your own time, then you have a bigger problem.
  • If conferences submissions are just extended abstracts, then we can focus our reviewing efforts on substantial papers.
  • At AMW2018, Paolo Guagliardo presented somebody else’s paper. He read the paper and did the best presentation possible which I’m sure will become a memorable presentation. Talking later to Hung Ngo and Paolo, we thought that it would be incredibly interesting to have a fellow colleague present your work. This could either be a PC member who reviewed your paper, or another colleague who shares the same interest and is willing to read the paper and present it. Imagine having somebody else being critical about your paper and present it to others. A bit risky for sure, but why not try this out with people who are willing to swap. Maybe at your next conference, you can surprise the audience with this approach. I know I would love to do it.

I acknowledge that my suggestion will never work due to the larger “system”. CS academics get evaluated by universities and funding agencies through the quantity of publications and citation counts. That has to be reformed. Easier said than done of course. If that continues to be the norm of evaluation, we are going to stay in the same place and keep adding bandaid after bandaid and hearing the same rants without any progressive change. Scientist high up in the ranks who have power are the ones that can make the change. I truly believe we need a change.

My 2 cents.