As you read through the guidelines and look at the papers in the annotation project of your choice, be ready to discuss the following questions. If a question is not answered in the material available or any other easily accessed material on the project (e.g. from a project site), just note that.
Overall your goal is to look at the annotation project from the perspectives of this class and the class deliverables.
1. What is the purpose of the annotation effort, that is, what larger effort will the results be used for (could just be to understand a particular aspect of language better or to provide models for an actual system)?
2. What is the annotation goal?
3. Describe the model (e.g. terms, relations, and interpretations).
4. Describe the corpus. How balanced and representative is it?
5. How well is the corpus suited to meeting the annotation goal?
6. Do the annotation guidelines describe the model in a clear and understandable examples? Does it have examples of both the canonical instances as well as the edge cases?
7. How objective/subjective are the decisions annotators are asked to make?
8. Using examples supplied (or examples from the unannotated corpus if any are readily available) how easy do you think the annotation is?