06 2014

Scientific Writing Beyond Peer Review

Felix Stalder

Let's start with a question: What is a publication project, situated at the intersection of theoretical and activist trajectories, supposed to achieve? One answer could be that it should serve as an organizing focus for a diverse community animated by the shared desire to advance its own distinct, systematic discursive practice. “Distinct” means that the project articulates a focus defined by overlapping frames of reference, areas of concern, and methods. The practice is “systematic” when each contribution places itself in relation to these parameters so that their transformation over time becomes matter of explicit debate. “Discursive practice” points to an understanding of theoretical debates as a form of social practice and vice versa. This is precisely what also calls for a greater diversity of the community, given the recognition of the multiple dimensions on which theoretical practice proceeds.

This is not a particularly controversial set of goals, but problems arise from the recognition that established publication models, particularly that of academic journals, rarely manage to achieve this. That failure is not accidental or due to issues of poor implementation of a conceptually sound model, but rooted in the very model of academic publishing itself. This model is largely unchanged since the days of the Royal Society, as are the epistemological assumptions that underlie it and the settlement they produce – namely the separation between science and politics.[1] In the following, I draw out some of these assumptions that are past their historical usefulness and suggest ways a contemporary publication model could proceed in order to have a better chance at achieving the goals just laid out.[2]

From Closed Institutions to Open Communities

Historically, abstract, systematic knowledge was contained within closed institutions: monasteries, guilds, universities, professional associations, research institutes, political parties and so on. To the degree that this knowledge was not commercial but still to be distributed, these institutions, organized since the 19^th century as modern bureaucracies divided into different departments, shared the costs of publication among themselves through fees paid to a learned society in the form of a subscription to a journal. Access felt free, as in no cost, to the people in these institutions who wrote and read these publications, as the subscription tended to be acceptably priced and was paid for through a dedicated budget, located elsewhere in the organization. That was a very effective system, as long as this assumption about the relationship between abstract knowledge and institutions was correct. As late as the 1970s, Daniel Bell imagined the future of the “post-industrial society” to be dominated by large institutions, populated by experts, as the sites of knowledge production.[3] This “venture in social forecasting”, as we all know now, turned out to be widely off the mark in this regard.

The production and use of abstract knowledge has broken out of the confines of the institutions and is now distributed across society at large. The conventional publication system has thus turned from an efficient, quasi free, distribution system into one that generates a sharp distinction between those still on the inside – where it continues to function more or less – and those who are on the outside of the institutional walled gardens, for whom this system turns into one of exclusion. This is not a question of the commercialization of the scientific publications, though this is a major issue in its own right, but of there being a price at all. For precarious knowledge workers who regularly sift through large quantities of information, even a small fee per article is prohibitive, let alone the absurdly high per-article fees usually charged. But it is not only a question of money. Surprisingly, still not all journals are on-line, and thus access to them, usually through a university library, requires at least having to physically travel a university building, which is in practice a substantial  effort for those who have no other connection to that university, producing a somewhat more subtle effect of exclusion.

The only way to minimize the distinction between those inside and those outside large institutions in terms of access is to publish as “open access”, which  many journals are doing now, following ten years of campaigning.[4] There is still a large number of journals that do not publish as open access, be it for reasons of commerce, prestige or institutional inertia, but the model is clearly established and there is absolutely no justification for not adopting it.

It is important that “open access” is not limited by hidden publication fees (again, small to those inside the institutions, prohibitive to those outside)[5] or by restrictive licenses that provide only access, but no reuse. We need to treat knowledge as social commons, from which community members take freely and to which they contribute in return. The most suitable license is a copyleft license,[6] which enables anyone to use the material without restrictions, under the one condition that she or he places the derived works back into the commons under the same conditions. This effectively makes it impossible to privatize knowledge as it evolves into new shapes and forms.

Peer Review as Forced Consensus

However, even the best open access model addresses only the most obvious problem of the current publication model: barrier to access based on price. It focuses solely on providing access to the finished text. It makes neither the process by which the text been shaped between submission and publication transparent, nor does it result in any changes to that process itself. Indeed, peer review still remains the core method for validating academic knowledge even in open access journals. In open access as well as pay-walled journals, this process is conventionally organized as “double blind”, meaning a submission is sent out to reviewers with author information removed, and the author receives the reviews in the same way. If the reviews are favorable, the author is supposed to address the points of criticism contained in the reviews and the revised article is published under the author's name. This creates a highly ideological and, in many ways, paradoxical object. There is a claim to impersonal knowledge (created through the review process) and there is the assertion of an individual authorial voice, by making the influence of the reviewers on the published text invisible.[7]

For all its historic merit, peer review today has become more problematic than ever. First, the abuses of the system which stifle publications that do not confirm the bias of the reviewers are legion.[8] Second, search engines and other tools make it easy to identify authors based on the style and subject matter of the text. Anonymity is hard to create these days. Third, peer review, even if it works well, tends to confirm the consensus in the field, formally and stylistically. Fourth, if it does not work well, which is very often the case, it is inconsistent, slow, and the reviewers' pettiness and vengefulness are shielded behind a veil of secrecy. For all these reasons, there are numerous proposals to reform the process of peer review. One of the most elaborate and radical has been formulated by Toni Prug as “open-process academic publishing”. Taking inspiration from Free Software and other open collaborative projects, Prug proposes to “open up”, all the steps of the review process by making it accessible through a public archive. In particular, this involves the “initial submissions, editorial collective and individual comments, peer reviews, further peer comments, author comments back to reviewers, all the subsequent drafts, and the final published or rejected text.”[9] In his view, the advantages of opening up the process in this way would be to reduce the abuses of the process, reveal inconsistencies and debates about how to resolve them, by creating accountability of all parties involved. It would also dramatically reduce the often long delays between submission and publication. In effect, each publication would be disaggregated into a pre-review version, the reviews and, if the reviews are favorable, the final post-review version. Technically, this requires some kind of “version control system”, something widely used in software development, but increasingly also in other fields.[10]
While this would likely reduce a number of the well-documented flaws in the peer review system, I'm skeptical that adding bureaucratic overhead can address the more fundamental flaws. It is quite possible that this would move the system even more towards consensus, as all parties try to minimize the risk of articulating marginal positions in a fully public system.

Thus, rather than trying to re-engineer the peer review process, we should contemplate abandoning the process altogether and the premises on which it is based: namely, that the advance of understanding proceeds through the creation of consensus and that it does so in discrete steps, each associated with an individual author and a stable statement of facts and references.

An Alternative Model Focusing on Debates

We should assume that knowledge emerges from debates and that debates, or discourses, are the central entities of knowledge production. So rather than trying to bring a text “up to standard” and then validating it with the stamp of peer review, contributions should be seen as starting points for debates that are open and open-ended. In this perspective, submissions should either be published as is, or rejected as is. But publication does not mean that the text or, more precisely, the debate that gives meaning to the text, is finished. On the contrary, it is only just starting.
In such a system, publication does not imply that the text conforms to a pre-established, impersonal standard that is self-explanatory. On the contrary, it indicates that at least someone in the editorial team decided that this text is worth publishing. And this decision needs to be argued. What is it, exactly, that makes this text, in the eyes of the editor(s), a worthwhile contribution. Consequently, a text submitted to a journal would not generate closed reviews to be incorporated in the text through invisible insertion, but rather a public response, open to further responses by other people, who may approach the text from an entirely different experience than the one taken for granted by the author and the reviewer or, now, the respondent. Debates about the interpretative framework through which a contribution to a commons field should be understood are to be expected and actively encouraged in diverse communities. Such differences and disagreements cannot be resolved by consensus, without establishing a privileged perspective and, in effect, homogenizing the community.

The task of an editorial board of a journal would therefore be two-fold: first, to assess whether a submitted contribution has the potential to advance the discursive practice of the community; second, the conclusion(s) of this evaluation by the editorial team are to be published as opening(s) of the debate alongside the initial contribution. The editorial team does not need to reach a consensus. It should be sufficient that one member can make a convincing case for the ways in which a contribution is, indeed, a contribution to the common discursive practice. Should other members come to different conclusions, their arguments should be published as well. It should only be rejected if nobody takes a positive interest.

This would dispense with the idea of knowledge as consensus. It would allow overcoming the paradox at the heart of the peer review system by enabling the author to really speak as him-/herself and, at the same time, shift the emphasis away from the single text to debate and collective discourse. Since we are living in an age of databases and search engines, we can seamlessly move across different scales. The trade-off between focusing on the discourse, and making the author's voice disappear into a murmur,[11] and focusing on the individual author and thus downplaying the role of the discourse is substantially reduced. We can foreground discourse and still make singular positions visible.

So, the emphasis of publication shifts away from the stable contribution from a single person to the more fluid and trans-formative debates generated within a field, driven forward by multiple contributions playing off each other, advancing and transforming a broad set of ideas and arguments. This would also allow moving materials more easily across different publication formats, from interventions to talks, perhaps recorded on video, to on-line articles to electronic and printed books and back again. There is no longer the need to pretend that each of them is a separate, distinct production, but one can think of them as different entry-points into, and passageways through, fields. Each of them has its unique quality, recognizing the heterogeneity of the community, each offering a particular discursive format and way of making an argument, pointing towards others’ contributions.

The various formats need to treated as equally important, rather than following the usual reputation hierarchy, in which the spoken word is ranked lowest and “reviewed” print highest. Such a hierarchy expresses the privileging of a particular culture – the academic print culture – over others. If the diversity of the community assembled around a publication project is to be taken seriously, this is not acceptable.

Providing open access on the basis of copyleft, replacing peer review with a system that promotes debate rather than forcing consensus, and building passageways between and across formats as a way to enter the discourse practice and the field generated by it, would be a way to move closer to the goals set by the project at the outset: to function as an organizing focus for a diverse community animated by the shared desire to advance a systematic discursive practice.

Language editing: Aileen Derieg