Want to take part in these discussions? Sign in if you have an account, or apply for one below
Vanilla 1.1.10 is a product of Lussumo. More Information: Documentation, Community Support.
This is an important question to figure out the answer to before determining a possible replacement. It’s also one that I doubt that there is going to be a single answer to. There will be many answers, and a robust replacement will have to take into account many of them. It’s also one that I don’t think that I have any glib answer to.
At the start of my mathematical career, I published in journals probably for two reasons:
Now, I’m not sure that either of those is really good enough. But I do still submit my articles to journals (when I can summon up the energy), so … why?
I think that the main reason is that when I write an article, it forces me to think properly about a problem. The way I work is quite fragmented and writing an article forces me to keep on track with a project. So I will often start the article fairly near the beginning of a project: I have quite a few nascent articles in my archives that probably will never see the light of day. Why do some survive and others wither? As I write and work, I get a sense of whether or not there is a “story” that is worth recording. If so, if it captures my imagination, then I’ll finish the article and try to get it published. At the least, I’ll put it on the arXiv.
For me, though, the main thing is not that others will read the article. I’m under no illusions that my articles are top of people’s reading lists and that there are hoards of researchers just waiting for my latest oeuvre. The main thing is that I write it as if someone else will read it, and again that forces me to write in a (hopefully) clear manner. The aim of that isn’t that I make the paper readable, but that if I’m not able to set it down in a way that I think someone else can read and understand, then I didn’t understand it in the first place.
So writing an article makes me think about the project in a manner that increases the chance of it turning from “I reckon I know how to prove this” into “I know how to prove this”. Writing an article as if for others to read then turns it into “I know how to prove this and here are the steps”.
But I could do all that without submitting an article to a journal. Except that I wouldn’t. I’m lazy and busy, two things that guarantee that if there isn’t someone standing over me with a big stick then things just won’t get done. I’m far more likely to write an article with a collaborator than one on my own simply because there’s someone else nagging me (not that they do!) to get it done.
There’s also an important distinction between posting on the arXiv and submitting to a journal. In the scene I’ve painted, posting on the arXiv is like saying “I’m ready for someone to take a look and see if I’ve missed anything”, whereas submitting to a journal is “I think I’ve got it”.
So what I’m looking for - as an author - in a replacement for journals is a replacement for that “Get on with it” factor. WIthout that, I’ll be awash with halfbaked articles that never quite get finished.
What else do we use journals for at the moment? What else would any replacement system need to provide?
This blog post lists five traditional functions of journals: Dissemination, Registration, Validation, Filtration, and Designation. Nowadays, I think the two strongest remaining rationales - due to a lack of plausible replacements - are validation (certifying correctness and significance of a paper) and designation (providing evidence of research achievement for the purposes of career advancement). The other three functions one can see being replaced by some variant of existing “Math 2.0” infrastructure, and for really significant papers we have seen ad hoc online efforts at validation (and, to a lesser extent, designation), but so far I don’t see how to scale these efforts so that a typical maths research paper gets vetted at a comparable level to what a typical maths journal currently provides, without basically having the functional equivalent of a journal.
Besides that list, and your own point about how the discipline of bringing a paper up to publishable standard helps improve the author’s research skills, is the feedback from the peer review process, which in my experience can sometimes improve the quality of a paper significantly. I definitely don’t want to abandon the peer review process as part of journal reform.
Andrew wrote:
But I could do all that without submitting an article to a journal. Except that I wouldn’t. I’m lazy and busy, two things that guarantee that if there isn’t someone standing over me with a big stick then things just won’t get done.
I agree. Lately I’ve been doing my research in the form of blog articles. It’s more fun, faster, and comparably good for getting the ideas out (though it wouldn’t be if I hadn’t spent decades building an audience). However, there’s a certain level of polishing and organizing that I don’t do unless I’m officially publishing something.
Some of this is habit, but some is the man with the big stick.
I think that regardless of what anyone says, the big remaining function served by journals is what teorth called ’designation’:
providing evidence of research achievement for the purposes of career advancement.
Until we figure out another way to do this, we’ll need journals.
but the notion that the multiple roles journals currently play could be replaced by a single model constructed to meet certain technical requirements is naive.
I completely agree. And whilst I may be naive, I’m not naive in that way. I’m not looking for a single model to replace journals. I’d prefer a much wider system, wherein each component of what journals currently provide is covered. Indeed, I think that one reason journals are such a bad system (I’m exaggerating a little) is because they try to do too much with one system. By separating out the components and figuring out a way of making each one work well, I believe we can get to a better overall setup. Moreover, by separating out the components we don’t have to replace journals all in one go but work bit by bit until eventually there’s no need for them anymore.
In particular, I agree with Joyal’s sentiment. When it comes to actually refereeing a paper, the paper should be at the best that the author can produce and it should be the referee’s job simply to referee, not to review or edit or search through for typos. Those things can be done first, and should be done first, so that the referee’s job is simple and clear. I think that one option for a referee really should be to say “This paper needs further polishing before it can be competently refereed” and that the referee should feel no obligation to do that polishing themselves because there is in place a mechanism (such as Tim Gowers’ “modest proposal”) where the author can go to get help with the polishing.
(Joyal’s comment also reminds me of a great article I once read about “Refereemanship”: gamesmanship applied to the refereeing process. Highly amusing, but also a little bit scary as I suspect that more of it goes on than ought to!)
Andrew, can you explain what distinction you draw between refereeing and reviewing? And what about peer review? (Is that the same as reviewing, in your book?) Or if you’ve already clarified it elsewhere, can you link to it?
To me, all those terms are essentially synonymous. (Of course, you can referee a football match and review a play, but….) So when you insist on the distinction between them, my reaction is the same as if someone insisted on a difference in meaning between jumper and sweater — they’re synonyms as far as I’m concerned, though they tend to be used by slightly different groups of people. I’m puzzled, but I’m sure you have a valid distinction to make, and I’d like to hear it.
Pure mathematicians always talk about “refereeing” papers, whereas some groups of applied mathematicians talk about “reviewing” them, as do some people outside mathematics. For example, I recently reviewed a theoretical paper for an ecology journal. The journal used the term reviewer rather than referee, but what I did for them was precisely the same as what I do when refereeing a pure math paper. The only obvious difference is that journals in that field tend to use two or three reviewers/referees, whereas pure mathematics journals seldom use more than one.
@Andrew - was it this: http://www.jstor.org/pss/2985643?
David: Yes! That’s the article I was thinking of. A must read for everyone here.
Tom, you’re right: I do see a distinction. Indeed, I think that there are lots of subcategories of “reviewing” that it would be beneficial to isolate because I think they get lumped together and it’s not always beneficial to do so.
In a simplified context, I see “refereeing” as a very focussed act. The referee is trying to answer the question “Should this article be published in this journal?”. To answer that, one would hope that the referee has been given a series of criteria to check (doubtful) which can be verified (even more doubtful) by examining the article. These should, of course, be public so that when an article appears in a particular journal, everyone can see what criteria that article can be assumed to have met.
“Reviewing” is far more diffuse, but one can divide it into two global parts: pre-publication and post-publication. In the pre-publication part, the goal of reviewing should be to help the author produce a publishable paper. This can involve checking the maths, checking the language, advising on lots of things. In the post-publication part, the goal of reviewing should be to help others decide whether or not they can use the paper, where “use” can mean anything from “read” to “rely on the results therein for their own work”. For this, there are no specific criteria because everyone who wants to use the review will have different purposes.
Others might not agree with my division of these into refereeing and reviewing - I don’t claim to have used those terms consistently myself - but where I would hope to find agreement is in that the divisions exist and that asking a referee to (in effect) play all of those roles is one aspect of what makes refereeing such an awful job to do.
Darij Greenberg wrote:
It just leads to tons of paper being wasted and, in the end, thrown away…
This is small and digressive point but: this is the second comment I’ve seen on this forum about ’wasted paper’ or ’conservation of paper’. Much as I’m in favor of efficiency and conservation, I have to say that the amount of paper used by scholarly journals is insignificant compared to the amount used in packaging, junk mail, and newspapers… and it’s far less of a ’waste’ than many of these other uses. So, given how many big questions we’re worrying about, I think waste of paper should not occupy our thoughts.
Yes, John, I was not making a point about environmental impact. What I was trying to say is that the belief that publication in a “proper” paper journal will immortalize the work is somewhat illusory.
but where I would hope to find agreement is in that the divisions exist and that asking a referee to (in effect) play all of those roles is one aspect of what makes refereeing such an awful job to do.
There might not be agreement even there. I just encountered this post, which makes the point that
I like that the person who’s being asked to evaluate my paper is also being asked to actually read it. … Even if the referees only point out a few typos and misspellings, it still prompts them to engage with the content of the paper, or at least to give it a try.
In other words, if the referee were asked only to decide “Should this article be published in this journal?”, it would make it more possible for them to make that decision based on facts that ought to be irrelevant, like the author’s reputation.
Put differently, someone being asked to decide whether this article should be published in this journal ought to be reading the paper closely enough to detect typos and tell whether the mathematics is correct anyway. Expecting them to do those other things as well is a way to (try to) ensure that they do read the paper that closely.
One might also make the point that the notion of “publishable paper” is not journal-independent, so it’s not possible to perform completely the process of “helping the author produce a publishable paper” independently of considering a particular journal for publication. That doesn’t necessarily mean that that process has to be combined with what you call refereeing, but since the referee is the one making the decision about whether the paper is publishable, surely they are the one best placed to help the author produce a paper satisfying that criterion!
Finally, I personally don’t find that refereeing is an “awful job”. Sometimes it’s a bit tiresome, but it’s good to be forced to read a paper outside my specialization carefully every so often, and it’s a good feeling when I help to improve someone else’s paper.
I guess maybe the larger point is that just because one system does many different jobs doesn’t necessarily mean that it would be more effective to do them separately with different systems. It could be that the jobs have been combined because it’s not possible to do some of them effectively on their own; and if that produces a somewhat unwieldy composite system, then that’s life.
Put differently, someone being asked to decide whether this article should be published in this journal ought to be reading the paper closely enough to detect typos and tell whether the mathematics is correct anyway. Expecting them to do those other things as well is a way to (try to) ensure that they do read the paper that closely.
I agree with the first sentence, and disagree with the second as it seems to be of the form “Good referees do proof-reading as an extra, so we make our referees good by getting them to do proof-reading”.
When I read a paper, I find it hard to concentrate so if I know I have to concentrate then I make myself proof-read it to keep my focus. Once I’ve done that, of course I’ll happily share the comments with the author. However, that’s a long way from requiring me to proof-read it if what you’re really asking me to do is evaluate it. I do agree with Izabella’s comment. I certainly want to know that a referee has actually read my paper - it’s not been my experience, though.
And that’s my real underlying gripe with the current system. It is so random. Some referees pick apart articles line by line, others take a vague look and think that’s enough. My first solo paper had a fantastic referee, but since then my experience has been … disappointing. I’ll not quote specifics, of course.
I also agree with your point about “publishable” being journal-specific (of course, as I’m trying to do away with journals then this needs considering as to how it would look in a new system), but I suspect that many articles could do with a lot of work that any journal would consider beneficial so then the referee’s comments would be a matter of refining the gem, not digging it out of the ground in the first place.
Whilst I grant that just because a system does many jobs doesn’t mean it necessarily is better to separate them, I must admit that my experience is that things tend to work better if each bit does one thing and does it really well. Moreover, even if it is clear that it is good to amalgamate, I’d rather have it that we decide to amalgamate them than it just happens organically. So even if the result is to say, “Actually, all these things should be done by the referee”, I still would like to know what “all these things” actually are.
While every of us seems to have his and her own goals and expectations for the SPN project, I think it is a good idea to not restrict it too much. It should be a place for discussions of papers from arXiv (and maybe even other databases - NUMDAM comes into my mind) which allows people to make remarks that wouldn’t make sense as standalone papers but have their worth nevertheless. These remarks can be referee reports, they can be lists of bugs, they can be “cheap” generalizations, and they can be questions of understanding arguments. And now that you (IL, post 19) are bringing this up, there might actually be a “this is a referee report 2.0” checkbox, which one can check if one has the appropriate privileges granted by the author and/or a circle of experts in that field, and which gives one’s comment significant weight (e. g., the rating in the comment contributes to a visible score, the comment appears at the beginning of the list, etc.). I certainly like this idea as long as it doesn’t become the only option to submit a comment.
David 12: I have no access, and my email to you bounced back, would you be so kind to send me the article ?
Sorry, Zoran! My university email changed a while back, but apart from updating it (in some places) I forgot to tell people… It is headed your way.
1 to 22 of 22