依存句法分析评价指标

依存句法分析器的性能主要可以从3个方面进行评价,分别是依存关系准确率(Dependency Accuracy, DA)、词根准确率(Root Accuracy, RA)以及完整句子依存结构准确率(Sentence Accuracy, SA)。
其中,考虑依存关系是否带有标签,又可以将DA分为带标签依存关系准确率(Labeled Attachment Score, LAS)和无标签依存关系准确率(Unlabeled Attachment Score, UAS)。

CoNLL对LAS和UAS的定义如下:
labeled attachment score: the proportion of “scoring” tokens that are assigned both the correct head and the correct dependency relation label.
Punctuation tokens are non-scoring. In very exceptional cases, and depending on the original treebank annotation, some additional types of tokens might also be non-scoring.
The overall score of a system is its labeled attachment score on all test sets taken together.

Unlabeled attachment score is the proportion of “scoring” tokens that are assigned the correct head (regardless of the dependency relation label).

DA = 正确预测的中心词的数量/预测的中心词数量
RA = 正确识别的根词数/根词总数
SA = 正确分析的句子数/句子总数
LAS = 正确预测中心词和标签的词的数量/总词数
UAS = 正确预测中心词的词的数量/总词数

<br> &nbsp; 蔡家勋<br><br> &nbsp; 计算机科学与工程系<br> &nbsp; 上海交通大学<br><br>