edu.illinois.cs.cogcomp.nlp.agreement

Class AnnotatorAgreement

java.lang.Object

edu.illinois.cs.cogcomp.nlp.agreement.AnnotatorAgreement

Direct Known Subclasses:

FleissMultiPi, PairwiseAgreement

public abstract class AnnotatorAgreement
extends Object

This abstract class and its descendants are based on the survey article "Inter-Coder Agreement for Computational Linguistics" by Ron Artstein and Massimo Poessio in Computational Lingusitics, Volume 34, Number 4.

Author:

Vivek Srikumar

Field Summary

Fields

Modifier and Type	Field and Description
protected int[][]	annotation
protected int[][]	nck
protected int[][]	nik
protected int[]	nk
protected int	numAnnotators
protected int	numItems
protected int	numLabels

Constructor Summary

Constructors

Constructor and Description
AnnotatorAgreement(int numAnnotators, int numItems, int numLabels)

Method Summary

Modifier and Type	Method and Description
void	addAnnotation(int annotatorId, int item, int label)
double	getAgreementCoefficient()
abstract double	getExpectedAgreement()
abstract double	getObservedAgreement()
double	getObservedDisagreement()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

numAnnotators

protected final int numAnnotators

numItems

protected final int numItems

numLabels

protected final int numLabels

nik

protected int[][] nik

nck

protected int[][] nck

nk

protected int[] nk

annotation

protected int[][] annotation

Constructor Detail

AnnotatorAgreement

public AnnotatorAgreement(int numAnnotators,
                          int numItems,
                          int numLabels)

Method Detail

addAnnotation

public void addAnnotation(int annotatorId,
                          int item,
                          int label)

getObservedAgreement

public abstract double getObservedAgreement()

getExpectedAgreement

public abstract double getExpectedAgreement()

getObservedDisagreement

public double getObservedDisagreement()

getAgreementCoefficient

public double getAgreementCoefficient()