edu.illinois.cs.cogcomp.lbjava.learn

Class AdaBoost

java.lang.Object

edu.illinois.cs.cogcomp.lbjava.classify.Classifier

edu.illinois.cs.cogcomp.lbjava.learn.Learner

edu.illinois.cs.cogcomp.lbjava.learn.AdaBoost

All Implemented Interfaces:

Serializable, Cloneable

public class AdaBoost
extends Learner

Implementation of the AdaBoost binary classification learning algorithm. This implementation samples from its internal distribution, giving the weak learner a new set of examples that the weak learner assumes are weighted equally.

Assumptions:

1. The weak learner is cloneable.
2. The weak learner is specified with the same values list as this learner.
3. The weak learning algorithm is trained on objects that are each given a single label feature.

Author:

Nick Rizzolo

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	AdaBoost.Parameters A container for all of AdaBoost's configurable parameters.

Field Summary

Fields

Modifier and Type	Field and Description
protected edu.illinois.cs.cogcomp.core.datastructures.vectors.OVector	allExamples All the examples observed by this learner during training.
protected String[]	allowableValues The label producing classifier's allowable values.
protected double[]	alpha Parameters associated with the trained copies of the weak learner.
static int	defaultRounds Default for rounds.
static Learner	defaultWeakLearner Default for weakLearner.
protected int	rounds The number of times the weak learner will be called.
protected Learner	weakLearner The weak learning algorithm to be boosted.
protected Learner[]	weakLearners Will be filled with trained copies of the weak learner.

Fields inherited from class edu.illinois.cs.cogcomp.lbjava.learn.Learner

candidates, encoding, extractor, labeler, labelLexicon, lcFilePath, lexFilePath, lexicon, lossFlag, predictions, readLexiconOnDemand

Fields inherited from class edu.illinois.cs.cogcomp.lbjava.classify.Classifier

containingPackage, name

Constructor Summary

Constructors

Constructor and Description
AdaBoost() Instantiates member variables.
AdaBoost(AdaBoost.Parameters p) Initializing constructor.
AdaBoost(int r) Instantiates member variables.
AdaBoost(Learner w) Instantiates member variables.
AdaBoost(Learner w, int r) Instantiates member variables.
AdaBoost(String n) Instantiates member variables.
AdaBoost(String n, AdaBoost.Parameters p) Initializing constructor.
AdaBoost(String n, int r) Instantiates member variables.
AdaBoost(String n, Learner w) Instantiates member variables.
AdaBoost(String n, Learner w, int r) Instantiates member variables.

Method Summary

Modifier and Type	Method and Description
String[]	allowableValues() Returns the array of allowable values that a feature returned by this classifier may take.
FeatureVector	classify(int[] exampleFeatures, double[] exampleValues) This method uses the trained parameters to make a binary decision about an example object.
String	discreteValue(int[] exampleFeatures, double[] exampleValues) This method uses the trained parameters to make a binary decision about an example object.
void	doneLearning() Performs learning on the examples stored in allExamples, if they exist; otherwise do nothing.
Feature	featureValue(int[] exampleFeatures, double[] exampleValues) Returns the classification of the given example as a single feature instead of a FeatureVector.
void	forget() Clears weakLearners and alpha, although this is not necessary since learn(Object[]) will overwrite them fresh each time it is called.
Learner.Parameters	getParameters() Retrieves the parameters that are set in this learner.
void	initialize(int numExamples, int numFeatures) Initializes the weight vector array to the size of the supplied number of features.
void	learn(int[] exampleFeatures, double[] exampleValues, int[] exampleLabels, double[] labelValues) This method adds the example object to the array storing the training examples.
void	learn(Object example) This method adds the example object to the array storing the training examples.
void	read(edu.illinois.cs.cogcomp.core.datastructures.vectors.ExceptionlessInputStream in) Reads the binary representation of a learner with this object's run-time type, overwriting any and all learned or manually specified parameters as well as the label lexicon but without modifying the feature lexicon.
ScoreSet	scores(int[] exampleFeatures, double[] exampleValues) Produces a set of scores indicating the degree to which each possible discrete classification value is associated with the given example object.
void	setLabeler(Classifier l) Sets the labeler.
void	setParameters(AdaBoost.Parameters p) Sets the values of parameters that control the behavior of this learning algorithm.
protected double[]	sumAlphas(int[] exampleFeatures, double[] exampleValues) Computes the scores corresponding to the two prediction values for the given example.
void	write(edu.illinois.cs.cogcomp.core.datastructures.vectors.ExceptionlessOutputStream out) Writes the learned function's internal representation in binary form.
void	write(PrintStream out) Writes this algorithm's internal representation as text.

Methods inherited from class edu.illinois.cs.cogcomp.lbjava.learn.Learner

classify, classify, classify, classify, clone, countFeatures, createPrediction, createPrediction, demandLexicon, discreteValue, discreteValue, doneWithRound, emptyClone, featureValue, featureValue, getCurrentLexicon, getExampleArray, getExampleArray, getExtractor, getLabeler, getLabelLexicon, getLexicon, getLexiconDiscardCounts, getLexiconLocation, getModelLocation, getPrunedLexiconSize, learn, learn, learn, read, readLabelLexicon, readLearner, readLearner, readLearner, readLearner, readLearner, readLearner, readLexicon, readLexicon, readLexiconOnDemand, readLexiconOnDemand, readModel, readModel, readParameters, realValue, realValue, realValue, save, saveLexicon, saveModel, scores, scores, scoresAugmented, setCandidates, setEncoding, setExtractor, setLabelLexicon, setLexicon, setLexiconLocation, setLexiconLocation, setLossFlag, setModelLocation, setModelLocation, setParameters, setReadLexiconOnDemand, unclone, unsetLossFlag, write, writeLexicon, writeModel, writeParameters

Methods inherited from class edu.illinois.cs.cogcomp.lbjava.classify.Classifier

classify, discreteValueArray, getCompositeChildren, getInputType, getOutputType, realValueArray, test, toString, valueIndexOf

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

defaultWeakLearner

public static final Learner defaultWeakLearner

Default for weakLearner.

defaultRounds

public static final int defaultRounds

Default for rounds.

See Also:

Constant Field Values

weakLearner

protected Learner weakLearner

The weak learning algorithm to be boosted.

rounds

protected int rounds

The number of times the weak learner will be called.

weakLearners

protected Learner[] weakLearners

Will be filled with trained copies of the weak learner.

alpha

protected double[] alpha

Parameters associated with the trained copies of the weak learner.

allExamples

protected edu.illinois.cs.cogcomp.core.datastructures.vectors.OVector allExamples

All the examples observed by this learner during training.

allowableValues

protected String[] allowableValues

The label producing classifier's allowable values.

Constructor Detail

AdaBoost

public AdaBoost()

Instantiates member variables.

AdaBoost

public AdaBoost(Learner w)

Instantiates member variables.

Parameters:

w - The weak learning algorithm.

AdaBoost

public AdaBoost(int r)

Instantiates member variables.

Parameters:

r - The number of rounds of boosting.

AdaBoost

public AdaBoost(Learner w,
                int r)

Instantiates member variables.

Parameters:

w - The weak learning algorithm.

r - The number of rounds of boosting.

AdaBoost

public AdaBoost(AdaBoost.Parameters p)

Initializing constructor. Sets all member variables to their associated settings in the AdaBoost.Parameters object.

Parameters:

p - The settings of all parameters.

AdaBoost

public AdaBoost(String n)

Instantiates member variables.

Parameters:

n - The name of the classifier.

AdaBoost

public AdaBoost(String n,
                Learner w)

Instantiates member variables.

Parameters:

n - The name of the classifier.

w - The weak learning algorithm.

AdaBoost

public AdaBoost(String n,
                int r)

Instantiates member variables.

Parameters:

n - The name of the classifier.

r - The number of rounds of boosting.

AdaBoost

public AdaBoost(String n,
                Learner w,
                int r)

Instantiates member variables.

Parameters:

n - The name of the classifier.

w - The weak learning algorithm.

r - The number of rounds of boosting.

AdaBoost

public AdaBoost(String n,
                AdaBoost.Parameters p)

Initializing constructor. Sets all member variables to their associated settings in the AdaBoost.Parameters object.

Parameters:

n - The name of the classifier.

p - The settings of all parameters.

Method Detail

setParameters

public void setParameters(AdaBoost.Parameters p)

Sets the values of parameters that control the behavior of this learning algorithm.

Parameters:

p - The parameters.

getParameters

public Learner.Parameters getParameters()

Retrieves the parameters that are set in this learner.

Overrides:

getParameters in class Learner

Returns:

An object containing all the values of the parameters that control the behavior of this learning algorithm.

allowableValues

public String[] allowableValues()

Returns the array of allowable values that a feature returned by this classifier may take.

Overrides:

allowableValues in class Classifier

Returns:

The allowable values of this learner's labeler, or an array of length zero if the labeler has not yet been established or does not specify allowable values.

setLabeler

public void setLabeler(Classifier l)

Sets the labeler.

Overrides:

setLabeler in class Learner

Parameters:

l - A labeling classifier.

initialize

public void initialize(int numExamples,
                       int numFeatures)

Initializes the weight vector array to the size of the supplied number of features.

Overrides:

initialize in class Learner

Parameters:

numExamples - The number of examples that will be observed during training.

numFeatures - The number of features that will be observed during training.

learn

public void learn(Object example)

This method adds the example object to the array storing the training examples.

Note that learning does not actually take place until doneLearning() is called.

Overrides:

learn in class Learner

Parameters:

example - The example object.

learn

public void learn(int[] exampleFeatures,
                  double[] exampleValues,
                  int[] exampleLabels,
                  double[] labelValues)

This method adds the example object to the array storing the training examples.

Note that learning does not actually take place until doneLearning() is called.

Specified by:

learn in class Learner

Parameters:

exampleFeatures - The example's array of feature indices.

exampleValues - The example's array of feature values.

exampleLabels - The example's label(s).

labelValues - The labels' values.

doneLearning

public void doneLearning()

Performs learning on the examples stored in allExamples, if they exist; otherwise do nothing.

Overrides:

doneLearning in class Learner

forget

public void forget()

Clears weakLearners and alpha, although this is not necessary since learn(Object[]) will overwrite them fresh each time it is called.

Overrides:

forget in class Learner

sumAlphas

protected double[] sumAlphas(int[] exampleFeatures,
                             double[] exampleValues)

Computes the scores corresponding to the two prediction values for the given example.

Parameters:

exampleFeatures - The example's array of feature indices.

exampleValues - The example's array of feature values.

Returns:

The scores corresponding to the values in the Learner.labelLexicon in an array with the same indexes.

scores

public ScoreSet scores(int[] exampleFeatures,
                       double[] exampleValues)

Produces a set of scores indicating the degree to which each possible discrete classification value is associated with the given example object.

Specified by:

scores in class Learner

Parameters:

exampleFeatures - The example's array of feature indices.

exampleValues - The example's array of feature values.

Returns:

The accumulated alpha values of weak learners that predicted the associated classification value.

featureValue

public Feature featureValue(int[] exampleFeatures,
                            double[] exampleValues)

Returns the classification of the given example as a single feature instead of a FeatureVector.

Overrides:

featureValue in class Learner

Parameters:

exampleFeatures - The example's array of feature indices.

exampleValues - The example's array of feature values.

Returns:

The classification of the example as a feature.

discreteValue

public String discreteValue(int[] exampleFeatures,
                            double[] exampleValues)

This method uses the trained parameters to make a binary decision about an example object.

Overrides:

discreteValue in class Learner

Parameters:

exampleFeatures - The example's array of feature indices.

exampleValues - The example's array of feature values.

Returns:

The decision value.

classify

public FeatureVector classify(int[] exampleFeatures,
                              double[] exampleValues)

This method uses the trained parameters to make a binary decision about an example object.

Specified by:

classify in class Learner

Parameters:

exampleFeatures - The example features.

exampleValues - The example values.

Returns:

A binary DiscreteFeature.

write

public void write(PrintStream out)

Writes this algorithm's internal representation as text.

Specified by:

write in class Learner

Parameters:

out - The output stream.

write

public void write(edu.illinois.cs.cogcomp.core.datastructures.vectors.ExceptionlessOutputStream out)

Writes the learned function's internal representation in binary form.

Overrides:

write in class Learner

Parameters:

out - The output stream.

read

public void read(edu.illinois.cs.cogcomp.core.datastructures.vectors.ExceptionlessInputStream in)

Reads the binary representation of a learner with this object's run-time type, overwriting any and all learned or manually specified parameters as well as the label lexicon but without modifying the feature lexicon.

Overrides:

read in class Learner

Parameters:

in - The input stream.