Show Summary Details

Page of

PRINTED FROM OXFORD HANDBOOKS ONLINE ( © Oxford University Press, 2018. All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a title in Oxford Handbooks Online for personal use (for details see Privacy Policy and Legal Notice).

date: 24 June 2021

Abstract and Keywords

In computational linguistics, humans or algorithms identify instances of some phenomenon in text and provide an interpretation of each instance. Usually the interpretations are labels chosen from a small set that reflect an underlying theory about the phenomenon. Setting up an annotation task requires expertise with several facets of the problem. First, characterising the phenomenon in terms of easily understandable labels and developing instructions for the annotators are important and often quite difficult steps. Next choosing the texts to be annotated, selecting the annotators, and training them are not fully resolved issues. A number of possibilities exist for evaluating their agreement. Existing tools, services, standards, and regular workshops dedicated to annotation are discussed.

Keywords: annotation, tagging, coding, annotators, Cohen’s kappa, inter-annotator agreement

Access to the complete content on Oxford Handbooks Online requires a subscription or purchase. Public users are able to search the site and view the abstracts and keywords for each book and chapter without a subscription.

Please subscribe or login to access full text content.

If you have purchased a print title that contains an access token, please see the token for information about how to register your code.

For questions on access or troubleshooting, please check our FAQs, and if you can''t find the answer there, please contact us.