edu.illinois.cs.cogcomp.lbjava.learn

Class BinaryMIRA

java.lang.Object

edu.illinois.cs.cogcomp.lbjava.classify.Classifier

edu.illinois.cs.cogcomp.lbjava.learn.Learner

edu.illinois.cs.cogcomp.lbjava.learn.LinearThresholdUnit

edu.illinois.cs.cogcomp.lbjava.learn.SparsePerceptron

edu.illinois.cs.cogcomp.lbjava.learn.BinaryMIRA

All Implemented Interfaces:

Serializable, Cloneable

public class BinaryMIRA
extends SparsePerceptron

The Binary MIRA learning algorithm implementation. This algorithm operates very similarly to SparsePerceptron with a thick separator, except the learning rate is a function of each training example's margin. When the weight vector has made a mistake, the full LinearThresholdUnit.learningRate will be used. When the weight vector did not make a mistake, LinearThresholdUnit.learningRate is multiplied by the following value before the update takes place.

(beta/2 - y(w*x)) / ||x||²

In the expression above, w is the weight vector, y represents the label of the example vector x, * stands for inner product, and beta is a user supplied parameter. If this expression turns out to be non-positive (i.e., if y(w*x) >= beta/2), no update is made for that example.

It is assumed that Learner.labeler is a single discrete classifier that produces the same feature for every example object and that the values that feature may take are available through the Classifier.allowableValues() method. The second value returned from Classifier.allowableValues() is treated as "positive", and it is assumed there are exactly 2 allowable values. Assertions will produce error messages if these assumptions do not hold.

This algorithm's user-configurable parameters are stored in member fields of this class. They may be set via either a constructor that names each parameter explicitly or a constructor that takes an instance of Parameters as input. The documentation in each member field in this class indicates the default value of the associated parameter when using the former type of constructor. The documentation of the associated member field in the Parameters class indicates the default value of the parameter when using the latter type of constructor.

Author:

Arindam Saha

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	BinaryMIRA.Parameters Simply a container for all of BinaryMIRA's configurable parameters.

Field Summary

Fields

Modifier and Type	Field and Description
protected double	beta The user supplied learning algorithm parameter; default defaultBeta.
static double	defaultBeta Default value for beta.
static double	defaultLearningRate Default value for LinearThresholdUnit.learningRate.
static double	TOLERANCE Used to decide if two values are nearly equal to each other.

Fields inherited from class edu.illinois.cs.cogcomp.lbjava.learn.LinearThresholdUnit

allowableValues, bias, defaultInitialWeight, defaultThickness, defaultThreshold, defaultWeightVector, initialWeight, learningRate, negativeThickness, positiveThickness, threshold, weightVector

Fields inherited from class edu.illinois.cs.cogcomp.lbjava.learn.Learner

candidates, encoding, extractor, labeler, labelLexicon, lcFilePath, lexFilePath, lexicon, lossFlag, predictions, readLexiconOnDemand

Fields inherited from class edu.illinois.cs.cogcomp.lbjava.classify.Classifier

containingPackage, name

Constructor Summary

Constructors

Constructor and Description
BinaryMIRA() The learning rate and beta take default values while the name of the classifier takes the empty string.
BinaryMIRA(BinaryMIRA.Parameters p) Initializing constructor.
BinaryMIRA(double r) Sets the learning rate to the specified value, and beta to the default, while the name of the classifier takes the empty string.
BinaryMIRA(double r, double B) Sets the learning rate and beta to the specified values, while the name of the classifier takes the empty string.
BinaryMIRA(double r, double B, SparseWeightVector v) Sets the learning rate, beta and the weight vector to the specified values.
BinaryMIRA(String n) Sets the name of the classifier to the specified value, while the learning rate and beta take default values.
BinaryMIRA(String n, BinaryMIRA.Parameters p) Initializing constructor.
BinaryMIRA(String n, double r) Sets the name of the classifier and learning rate to the specified values, while beta takes the default value.
BinaryMIRA(String n, double r, double B) Sets the name of the classifier, the learning rate and beta to the specified values.
BinaryMIRA(String n, double r, double B, SparseWeightVector v) Sets the name of the classifier, the learning rate, beta and the weight vector to the specified values.

Method Summary

Modifier and Type	Method and Description
double	computeLearningRate(int[] exampleFeatures, double[] exampleValues, double s, boolean label) Computes the learning rate for this example.
void	demote(int[] exampleFeatures, double[] exampleValues, double rate) Scales the feature vector produced by the extractor by the learning rate and subtracts it from the weight vector.
double	getBeta() Returns the current value of the beta member variable.
double	getLearningRate() Returns the original value of the LinearThresholdUnit.learningRate variable.
Learner.Parameters	getParameters() Retrieves the parameters that are set in this learner.
void	promote(int[] exampleFeatures, double[] exampleValues, double rate) Scales the feature vector produced by the extractor by the learning rate and adds it to the weight vector.
void	read(edu.illinois.cs.cogcomp.core.datastructures.vectors.ExceptionlessInputStream in) Reads the binary representation of a learner with this object's run-time type, overwriting any and all learned or manually specified parameters as well as the label lexicon but without modifying the feature lexicon.
void	setBeta(double B) Sets the beta member variable to the specified value.
void	setParameters(BinaryMIRA.Parameters p) Sets the values of parameters that control the behavior of this learning algorithm.
boolean	shouldDemote(boolean label, double s, double threshold, double negativeThickness) Determines if the weights should be promoted.
boolean	shouldPromote(boolean label, double s, double threshold, double positiveThickness) Determines if the weights should be promoted.
void	write(edu.illinois.cs.cogcomp.core.datastructures.vectors.ExceptionlessOutputStream out) Writes the learned function's internal representation in binary form.
void	write(PrintStream out) Writes the algorithm's internal representation as text.

Methods inherited from class edu.illinois.cs.cogcomp.lbjava.learn.SparsePerceptron

setLearningRate

Methods inherited from class edu.illinois.cs.cogcomp.lbjava.learn.LinearThresholdUnit

allowableValues, classify, clone, discreteValue, featureValue, forget, getAllowableValues, getBias, getInitialWeight, getNegativeThickness, getPositiveThickness, getThreshold, getWeightVector, initialize, learn, score, score, scores, setInitialWeight, setLabeler, setNegativeThickness, setParameters, setPositiveThickness, setThickness, setThreshold

Methods inherited from class edu.illinois.cs.cogcomp.lbjava.learn.Learner

classify, classify, classify, classify, countFeatures, createPrediction, createPrediction, demandLexicon, discreteValue, discreteValue, doneLearning, doneWithRound, emptyClone, featureValue, featureValue, getCurrentLexicon, getExampleArray, getExampleArray, getExtractor, getLabeler, getLabelLexicon, getLexicon, getLexiconDiscardCounts, getLexiconLocation, getModelLocation, getPrunedLexiconSize, learn, learn, learn, learn, read, readLabelLexicon, readLearner, readLearner, readLearner, readLearner, readLearner, readLearner, readLexicon, readLexicon, readLexiconOnDemand, readLexiconOnDemand, readModel, readModel, readParameters, realValue, realValue, realValue, save, saveLexicon, saveModel, scores, scores, scoresAugmented, setCandidates, setEncoding, setExtractor, setLabelLexicon, setLexicon, setLexiconLocation, setLexiconLocation, setLossFlag, setModelLocation, setModelLocation, setParameters, setReadLexiconOnDemand, unclone, unsetLossFlag, write, writeLexicon, writeModel, writeParameters

Methods inherited from class edu.illinois.cs.cogcomp.lbjava.classify.Classifier

classify, discreteValueArray, getCompositeChildren, getInputType, getOutputType, realValueArray, test, toString, valueIndexOf

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

TOLERANCE

public static final double TOLERANCE

Used to decide if two values are nearly equal to each other.

See Also:

nearlyEqualTo(double,double), Constant Field Values

defaultBeta

public static final double defaultBeta

Default value for beta.

See Also:

Constant Field Values

defaultLearningRate

public static final double defaultLearningRate

Default value for LinearThresholdUnit.learningRate.

See Also:

Constant Field Values

beta

protected double beta

The user supplied learning algorithm parameter; default defaultBeta. The learning rate changes as a function of beta.

Constructor Detail

BinaryMIRA

public BinaryMIRA()

The learning rate and beta take default values while the name of the classifier takes the empty string.

BinaryMIRA

public BinaryMIRA(double r)

Sets the learning rate to the specified value, and beta to the default, while the name of the classifier takes the empty string.

Parameters:

r - The desired learning rate value.

BinaryMIRA

public BinaryMIRA(double r,
                  double B)

Sets the learning rate and beta to the specified values, while the name of the classifier takes the empty string.

Parameters:

r - The desired learning rate value.

B - the desired beta value.

BinaryMIRA

public BinaryMIRA(double r,
                  double B,
                  SparseWeightVector v)

Sets the learning rate, beta and the weight vector to the specified values.

Parameters:

r - The desired learning rate.

B - The desired beta value.

v - The desired weight vector.

BinaryMIRA

public BinaryMIRA(BinaryMIRA.Parameters p)

Initializing constructor. Sets all member variables to their associated settings in the BinaryMIRA.Parameters object.

Parameters:

p - The settings of all parameters.

BinaryMIRA

public BinaryMIRA(String n)

Sets the name of the classifier to the specified value, while the learning rate and beta take default values.

Parameters:

n - The name of the classifier.

BinaryMIRA

public BinaryMIRA(String n,
                  double r)

Sets the name of the classifier and learning rate to the specified values, while beta takes the default value.

Parameters:

n - The name of the classifier.

r - The desired learning rate.

BinaryMIRA

public BinaryMIRA(String n,
                  double r,
                  double B)

Sets the name of the classifier, the learning rate and beta to the specified values.

Parameters:

n - The name of the classifier.

r - The desired learning rate.

B - The desired beta value.

BinaryMIRA

public BinaryMIRA(String n,
                  double r,
                  double B,
                  SparseWeightVector v)

Sets the name of the classifier, the learning rate, beta and the weight vector to the specified values. Use this constructor to specify an alternative subclass of SparseWeightVector.

Parameters:

n - The name of the classifier.

r - The desired learning rate.

B - The desired beta value.

v - The desired weight vector.

BinaryMIRA

public BinaryMIRA(String n,
                  BinaryMIRA.Parameters p)

Initializing constructor. Sets all member variables to their associated settings in the BinaryMIRA.Parameters object.

Parameters:

n - The name of the classifier.

p - The settings of all parameters.

Method Detail

setParameters

public void setParameters(BinaryMIRA.Parameters p)

Sets the values of parameters that control the behavior of this learning algorithm.

Parameters:

p - The parameters.

getParameters

public Learner.Parameters getParameters()

Retrieves the parameters that are set in this learner.

Overrides:

getParameters in class SparsePerceptron

Returns:

An object containing all the values of the parameters that control the behavior of this learning algorithm.

getBeta

public double getBeta()

Returns the current value of the beta member variable.

Returns:

The value of the beta variable.

setBeta

public void setBeta(double B)

Sets the beta member variable to the specified value.

Parameters:

B - The new value for beta.

shouldPromote

public boolean shouldPromote(boolean label,
                             double s,
                             double threshold,
                             double positiveThickness)

Determines if the weights should be promoted.

Overrides:

shouldPromote in class LinearThresholdUnit

Parameters:

label - The label of the example object.

s - The score of the example object.

threshold - The LTU threshold.

positiveThickness - The thickness of the hyperplane on the positive side.

Returns:

true iff the weights should be promoted.

shouldDemote

public boolean shouldDemote(boolean label,
                            double s,
                            double threshold,
                            double negativeThickness)

Determines if the weights should be promoted.

Overrides:

shouldDemote in class LinearThresholdUnit

Parameters:

label - The label of the example object.

s - The score of the example object.

threshold - The LTU threshold.

negativeThickness - The thickness of the hyperplane on the negative side.

Returns:

true iff the weights should be demoted.

promote

public void promote(int[] exampleFeatures,
                    double[] exampleValues,
                    double rate)

Scales the feature vector produced by the extractor by the learning rate and adds it to the weight vector.

Overrides:

promote in class SparsePerceptron

Parameters:

exampleFeatures - The example's array of feature indices.

exampleValues - The example's array of feature values.

rate - The learning rate at which the weights are updated.

demote

public void demote(int[] exampleFeatures,
                   double[] exampleValues,
                   double rate)

Scales the feature vector produced by the extractor by the learning rate and subtracts it from the weight vector.

Overrides:

demote in class SparsePerceptron

Parameters:

exampleFeatures - The example's array of feature indices.

exampleValues - The example's array of feature values.

rate - The learning rate at which the weights are updated.

computeLearningRate

public double computeLearningRate(int[] exampleFeatures,
                                  double[] exampleValues,
                                  double s,
                                  boolean label)

Computes the learning rate for this example.

Overrides:

computeLearningRate in class LinearThresholdUnit

Parameters:

exampleFeatures - The example's array of feature indices.

exampleValues - The example's array of feature values.

s - The score for the example object.

label - The label.

Returns:

The new learning rate.

getLearningRate

public double getLearningRate()

Returns the original value of the LinearThresholdUnit.learningRate variable.

Overrides:

getLearningRate in class SparsePerceptron

Returns:

The value of the LinearThresholdUnit.learningRate variable.

write

public void write(PrintStream out)

Writes the algorithm's internal representation as text. In the first line of output, the name of the classifier is printed, followed by LinearThresholdUnit.learningRate, beta, LinearThresholdUnit.initialWeight, LinearThresholdUnit.threshold, LinearThresholdUnit.positiveThickness, LinearThresholdUnit.negativeThickness, and finally LinearThresholdUnit.bias.

Overrides:

write in class SparsePerceptron

Parameters:

out - The output stream.

write

public void write(edu.illinois.cs.cogcomp.core.datastructures.vectors.ExceptionlessOutputStream out)

Writes the learned function's internal representation in binary form.

Overrides:

write in class LinearThresholdUnit

Parameters:

out - The output stream.

read

public void read(edu.illinois.cs.cogcomp.core.datastructures.vectors.ExceptionlessInputStream in)

Reads the binary representation of a learner with this object's run-time type, overwriting any and all learned or manually specified parameters as well as the label lexicon but without modifying the feature lexicon.

Overrides:

read in class LinearThresholdUnit

Parameters:

in - The input stream.