Annotation Guidelines Analysis

By | January 6, 2018

Below are a guidelines from a number of different annotation efforts. Groups will discuss a set of guidelines and answer specific questions then report back to the class on what you learned.

Step 1, Jan 16: Each person will choose one project to focus on. You will then form groups of 3 and group members will review the annotation guidelines the other group members (we’ll handle any duplicates in class).

Step 2, Jan 19: Groups will spend part of the class answering a set of questions about each project and comparing the different approaches.

Step 3: Jan 23: Each group will present what they’ve learned to the rest of the class. Presentations should be roughly 10 minutes.

Annotation Efforts

ICSI Meeting Recorder corpus

Dialog Act Guidelines
Paper on ICSI Meeting Recorder Dialog Act (MRDA) Corpus
Project Site

AMI Meeting Corpus

Topic Segmentation Guidelines
AMI Meeting Corpus Overview
Other AMI Annotation Guidelines

Settlers of Catan

STAC Annotation Guidelines
Discourse Structure and Dialogue Acts in Multiparty Dialogue

ARRAU: Anaphora and Deictic reference in dialogs

ARRAU Annotation Guidelines (for TRAINS)
Anaphoric Annotation in the ARRAU Corpus

SemEval 2016 Task 5 Aspect Based Sentiment Analysis (ABSA-16)

Overview Paper
Workshop Proceedings
Project Site

Penn Discourse TreeBank

Penn Discourse Treebank Annotation Manual
Overview paper
Publication list

THYME: Temporal Histories of Your Medical Events

THYME Annotation Guidelines
Temporal Annotation in the Clinical Domain
THYME project overview


TimeMLAnnotation Guidelines
Temporal Discourse Models for Narrative Structure
TimeML Project Site


SpaceML Guidelines
Imace Annotation with ISO-SPace

Committed Belief

Annotation Belief in Communication: Manual
Committed Belief Annotation and Tagging