I was recently appointed co-editor-in-chief (EIC) of the Information Retrieval Journal, a long-standing Springer publication focused on the theory, algorithms, analysis, and evaluation of search systems. The information retrieval (IR) field draws on research from computer science and includes contributions from other areas, such as artificial intelligence, information science, psychology, and human factors. IR technologies have formed the basis of popular web search engines, which are essential tools to help people find and understand information. The IR Journal receives hundreds of submissions per year. Since its inception in 1999, the journal has published over 500 technical papers on many different aspects of IR research.[1]
In this article, I discuss some of the unique aspects of computer science publishing and how my experience as an author and a reviewer has helped guide me in my new role as co-EIC of the IR Journal. Given the rapid pace of progress in computer science, conference presentations and proceedings are the primary means of disseminating scientific findings in that area. Journals are often used to publish extended versions of proceedings papers with expanded experiments, additional discussion, and more detailed literature reviews. Looking to the future, I would like to see computer scientists more fully embrace journals asa viable alternative to conferences and a venue to present their latest significant research contributions. Expediting peer reviewing and online publication, boosting visibility via social media channels, and partnering with conferences to offer presentation opportunities for select articles are all ways to help journals thrive and grow in a fast-paced research discipline such as computer science.
Reviewing Experience
I have been an active researcher in IR for almost 20 years. During that time, I have published hundreds of technical publications, including conference papers, journal articles, and textbooks. My activities as a highly active author have provided me with significant appreciation for the author’s perspective in the publication process. This includes understanding the importance of critical and constructive reviewer feedback and acknowledging the significant role that editors play in shaping the research landscape through the decisions they make and the guidance they offer.
As is the case for many scientists, I have been an active peer reviewer of research publications since early in my research career. This has afforded me an opportunity to provide plenty of constructive feedback on the research of my peers (a skill that takes time to develop) and learn about reviewing best practices and pitfalls from observing others in action and participating in discussions with them. This has been invaluable training for the practice of science in general and for the EIC role.
One topic that is consistently important for both authors and journal editors is expectation management. As an editor, I spend considerable time ensuring (through written communications) that there is a shared understanding between myself and authors about the state of their submission and its trajectory toward possible publication. In my experience, failure to communicate will inevitably lead to frustration for authors and wasted effort for both parties. I often desk-reject submissions that are off-topic or have a low chance of becoming a manuscript that is likely to be accepted in a reasonable timeframe. I accompany each desk rejection with an explanatory note tailored to the specific article. For each submission under review, I provide a detailed meta-review at each iteration that includes the relative priority of the changes requested by reviewers, which can be diverse and voluminous.
Onboarding
Joining a journal as a new EIC can be overwhelming. Support from the journal editorial office and any co-EICs is vital to assimilate quickly. Tacit knowledge about journal practices and the state of the journal need to be shared promptly by the journal editorial office (JEO) and outgoing editor(s). It is important that authors with submissions in progress not be impacted by the change in EIC: Review flow should be uninterrupted, and submissions need to be carefully tracked so they are not misplaced. As I joined the IR Journal, we opted to have the outgoing EIC manage all in-review submissions through the final decision as a way to maintain consistency and reduce the chance of error. Managing such legacy duties is part of a set of possible challenges for departing EICs.
It is common for new EICs to want to improve aspects of the journal; in my case, this included growing the social media presence of the IR Journal, reducing article processing times, and improving the visibility of published articles. The IR Journal has 3 EICs: one for the Americas (me), one for Europe, and one for Asia and rest of the world. It was vital to align quickly with my co-EICs and the JEO on each objective, and to get support and feedback from them. Working with the JEO and my co-EICs, we are implementing some changes in journal practice to meet each goal (and making other improvements). I discuss some of these modifications in the remainder of this article.
Shared Editorship
Having multiple EICs means that decisions are made by consensus, which can add overhead, but it has clear advantages including offering different perspectives and shared workload. Varying perspectives are important for the IR Journal because the field of IR is multidisciplinary, and we want to ensure that each perspective is considered in our discussions and decisions. In terms of workload, at present each editor manages the submissions from his or her assigned geographic region. The geographic split is helpful for many reasons (e.g., fielding author questions in a timely manner). However, we also find that the number of submissions is skewed toward specific regions; this creates significantly more work for certain editors. The EICs and JEO are revisiting the regional strategy and exploring alternatives to ensure that we distribute responsibilities more fairly among the EICs (e.g., using round-robin or time-boxed solutions to assign EICs to incoming submissions).
Publication Models
Computer science research is published both at conferences and in journals, with conference papers (6000–10,000 words in length) often serving as the initial, and quite often the primary, publication venue. The focus on conference proceedings can largely be attributed to the rapid innovation in computer science and the need to publish research results quickly to keep pace. Some of the changes in journal publication practice, such as “early online” publishing, may encourage more computer scientists to consider journals as an initial outlet. However, this needs to be accompanied by high impact factors, high visibility, and quick turnaround times for reviewing—all are significant challenges for editors.
Computer science journals publish new work; but quite frequently, they publish extended versions of conference proceedings papers. Although the journal version of a proceedings paper may have considerable additions to the original paper (25 50% new material in many cases), these additions often simply comprise more experiments, discussion, and related work, but less in terms of novel technical contribution. As a result, the proceedings paper often remains the canonical reference for the research.
Computer science journals must endeavor to drive journal awareness and highlight the advantages of publishing there first (including near unlimited space for authors to discuss their research and an opportunity to improve publication quality through multiple rounds of review). This ranges from lightweight methods with broad reach (e.g., advertising new articles on social media) to deeper collaborations with specific conferences, which provides authors with a forum to present their research, obtain increased visibility, and have more impact with the work.
To this end, the IR Journal has recently partnered with the European Conference on Information Retrieval (ECIR), whose proceedings are also published by Springer, to provide a source of high-quality publications for the IR Journal and provide our authors with greater visibility for their published articles. Authors of select articles from the IR Journal in the previous calendar year will be invited to present at the next ECIR, and authors of award-winning/top-ranked ECIR papers will be invited to submit extended versions of their conference papers for fast-track review and possible publication in the IR Journal. Although theseare extended versions of conference papers, and not purely original work (my ideal outcome, as highlighted earlier), I believe that there is considerable value in pursuing this to forge a relationship with ECIR, grow awareness of the IR Journal, and provide authors with an additional incentive to submit to the conference. Other computer science journals adopt a similar model, albeit asymmetric; they only provide the chance to present journal articles at conferences but do not also provide journal publication opportunities for proceedings authors. The Association for Computing Machinery’s (ACM) Transactions on Information Systems (TOIS) journal allows authors to present their papers at several ACM conferences, and Transactions for the Association for Computational Linguistics is a journal that has a monthly deadline and conference presentation opportunities.
Impact and Reach
An important determinant of a journal’s success is the impact of the research it publishes. Although improving impact scores can create a virtuous cycle of high-quality submissions, low scores can be difficult to change. In my experience, EICs often need to be creative in taking steps to improve the visibility of existing articles and attract new articles that are likely to be well cited (e.g., invite articles from specific senior scientists or arrange special issues of the journal on timely topics).
Computer science journals range from specialized, highly technical publications such as the IR Journal, to more mainstream publications, such as Communications of the ACM (CACM). The ACM is the flagship computing society in the United States. CACM is distributed to all ACM members and articles are meant to be accessible and relevant to a broad readership. Several other journals—such as ACM TOIS, mentioned above—also publish high-quality, highly technical IR research. Healthy competition forces journals to devise creative publishing models, reduce turnaround times, and improve the quality and visibility of the work they publish; these are all helpful for authors and the research community at large.
Review Process
One of the main bottlenecks for journals is finding willing reviewers and ensuring that they submit reviews on time. Tardiness in replying to invitations or completing reviews slows down the review process considerably. The IR Journal has a standing editorial board of senior researchers who serve as a source of reviewers for submissions that have already passed an initial check for relevance and technical depth, performed by the EICs. Having a standing board of senior experts helps alleviate reviewing delays and helps ensure that the reviews are consistent across submissions and of high quality.
Full utilization of the IR Journal editorial board requires careful planning. As EICs, we want to ensure that editorial board members are aware of incoming submissions but also do not want to overwhelm them with an invite for each submission, and EICs do not want to spend considerable time matching submissions to specific reviewers. At the IR Journal, we have recently arrived at a middle ground where we periodically distribute a digest of pending submissions (every 1–2 weeks), with a request for volunteers from the editorial board with interest and expertise to review them. Although this model works well most of the time, we still send directed review requests to specific reviewers if we require their input, given their expertise.
Beyond scoring received submissions, reviewers are the front line on detecting copyright issues, such as articles submitted or already published elsewhere. They are also an imperfect sensor, dependent largely on their memory, general awareness of content presented at other venues, and their attention to detail. Plagiarism detection software can also catch some cases. As with many journals, the IR Journal follows a clear policy—the Committee on Publishing Ethics standards—regarding plagiarism detection and follow-up. However, such cases consume considerable time and I believe that we need to do more in the computer science community to actively discourage dual submissions. Some computer science conferences now ban dual-submitting authors from submitting to the same conference in the future. Although I focus on computer science publishing, these and similar challenges are faced by any scientific discipline where there is time pressure on authors to publish their research findings quickly.
Experience So Far
I find the EIC role to be incredibly rewarding. The role carries with it considerable responsibility to ensure that the IR Journal publishes high-quality research that interests our readership and has impact in both the short and long term. I would advise those considering an EIC opportunity to give serious thought as to whether they have the time and enthusiasm to serve as an effective EIC. Even if the position is shared with co-EICs, I feel that it is important to be fully invested to maximize the opportunity to help shape the field. Most importantly, EICs need to be supportive and fair to authors who entrust them with their research, and in many cases, their careers.
For Authors
For those looking to find a journal with which to publish, I would suggest reading several prior articles from your target journal to obtain a sense of the style and relevance to the subject matter of your work; additionally, consider approaching the EIC(s) with an inquiry about whether the work is appropriate and worth submitting. In my experience as an author, I have found that it is best to do this before going through the (often considerable) effort of reformatting the paper, pruning or adding words to meet word limits, writing cover letters, and so on. EICs are typically responsive to requests and they are often willing to provide feedback on whether submitting is worth your time (e.g., whether a manuscript is likely to at least be peer reviewed).
In the future, I hope to see journals play a larger role in the dissemination of computer science research, and I am doing what I can as an EIC to provide a supportive and impactful publication venue.
Ryen W. White, PhD, is a Partner Researcher and Research Manager at Microsoft Research, Redmond, Washington.
[1] Recent metrics for the IR Journal: acceptance rate: 8%, h-index: 54, impact factor: 1.488.