Assignment 2: Annotation of dialog and reviews
Each group will be assigned an annotation effort from the set below. Read the paper and look through the guidelines. We’ll provide you some data to look at before Friday’s class. Your goal in class will be to try to annotate that data given the guidelines and create a presentation covering the goal of the annotation, a description and assessment of the guidelines (including your experience trying to follow them), an a quick summary of the evaluation results and what kinds of machine learning algorithms were tried.
Presentations on Tuesday, March 7
- about 10-15 mins.
- Annotation Goal
- Task Description
- Description of corpus
- Annotation Guidelines and your experience applying it
- Results: inter annotator agreement, evaluation results, 1-2 examples of ML algorithms used on the data (not all of these will be available for every project. Just include what you can find)
- What you learned that applies to your project
SemEval 2014 Task 4: Aspect Based Sentiment Analysis
Overview Paper
Guidelines
Workshop Proceedings
Project Site
SemEval 2016 Task 5 Aspect Based Sentiment Analysis (ABSA-16)
Overview Paper
Guidelines
Workshop Proceedings
Project Site
ICSI corpus
Overview of the corpus
Paper about the CALO project
Project Site
Dialog Acts
Paper on ICSI Meeting Recorder Dialog Act (MRDA) Corpus
Dialog Act Guidelines
Paper on Dialog Act classification across multiple corpora
Topic Segmentation
Topic Annotation Guidelines
Paper on topic segmentation
Paper From AMI corpus work, but it’s pretty good.