Collecting reader feedback with PaperHive, docLoop and GitHub

Back in 2017, we wrote a blog post on fluid publication. This explained the development of a book by the author together with the readership, reusing techniques well-known from software development.

The author 1) starts with a draft version, collects feedback from colleagues, and then the stages of 2) (open) review, 3) acceptance, 4) community proofreading and finally 5) publication of the first edition follow. A history of the different versions is kept on GitHub. GitHub also provides functionalities to manage lists of open issues which still have to be addressed before the next stage can be initiated.

As detailed in various posts on this blog, we use PaperHive for community proofreading. Today, we can showcase docLoop, which allows us to transform the community comments into todo lists on GitHub, closing the loop from author to reader and back from the reader to the author.

Let’s look at an example, Voice at the interfaces: The syntax, semantics, and morphology of the Hebrew verb by Itamar Kastner. We can see the progress of this book on its GitHub page. The book was started in June 2018 and finalised in June 2020. Between the setup of the project and the publication, we count 259 different versions. Next to the author itamarkast, who provided 227 improved versions, kopeckyf and Glottotopia from the LangSci team provided 24 and 3 commits, respectively.

Between the plan of the book, created on May 17, 2018, and the final publication in June 2020, the book grew from an outline to a respectable 286 page volume. The first full draft was completed on July 24, 2019. Between August and November 2019, we see a period of no activity, as the manuscript was under review then.

After the note of acceptance comes in in November, editing starts again until the manuscript reaches the community proofreading stage on April 22. Community proofreading is done via PaperHive. A total of 626 comments were left by the community for this book. The following shows two comments by Bev Erasmus and Tom Bossuyt, commenting on consistency and typography.

Our traditional workflow requires that authors go through the PaperHive document and take care of the comments as they go along, updating their manuscript. With docLoop, however, we can now harvest all those comments and put them in a nice GitHub issue list. The comment from Bev Erasmus above was harvested on May 22, addressed by the author on May 23 and marked as resolved (“closed”) on May 27

The availability of all issues in a structured list allows for neat sorting and triaging. For instance, the author created the tag “help wanted” for issues he could not deal with himself. The author added “help wanted” to this issue on May 24, and the LangSci office resolved the problem on June 3 and closed the issue.

Iterative publication does not end with the first edition, as explained in our 2017 blogpost. Readers will have feedback, and Paperhive allows us to collect this feedback. Corpus linguistics: A guide to the methodology by Anatol Stefanowitsch was published in April 2020. On its Paperhive page, readers have left comments, which have been imported into GitHub via docLoop so that these can be addressed for subsequent editions.

Readers do not have to learn anything about git, though. They can simply use the very user-friendly PaperHive web interface to leave their feedback. This feedback is then converted into structured issues for further processing. The following chart visualises the full loop.

The development of docLoop was financed by the ProtoType fund, a great initiative which funds technologies for a more open and more inclusive world.

Leave a Reply

Your email address will not be published.

Hinweis / Hint
Das Captcha kann Kleinbuchstaben, Ziffern und die Sonderzeichzeichen »?!#%&« enthalten.
The captcha could contain lower case, numeric characters and special characters as »!#%&«.