edu.illinois.cs.cogcomp.lbjava.learn

Class SparseWinnow

java.lang.Object

edu.illinois.cs.cogcomp.lbjava.classify.Classifier

edu.illinois.cs.cogcomp.lbjava.learn.Learner

edu.illinois.cs.cogcomp.lbjava.learn.LinearThresholdUnit

edu.illinois.cs.cogcomp.lbjava.learn.SparseWinnow

All Implemented Interfaces:

Serializable, Cloneable

public class SparseWinnow
extends LinearThresholdUnit

Simple sparse Winnow implementation. It is assumed that Learner.labeler is a single discrete classifier whose returned feature values are available through the Classifier.allowableValues() method. The second value returned from Classifier.allowableValues() is treated as "positive", and it is assumed there are exactly 2 allowable values. Assertions will produce error messages if these assumptions do not hold.

This algorithm's user-configurable parameters are stored in member fields of this class. They may be set via either a constructor that names each parameter explicitly or a constructor that takes an instance of Parameters as input. The documentation in each member field in this class indicates the default value of the associated parameter when using the former type of constructor. The documentation of the associated member field in the Parameters class indicates the default value of the parameter when using the latter type of constructor.

Author:

Nick Rizzolo

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	SparseWinnow.Parameters Simply a container for all of SparseWinnow's configurable parameters.

Field Summary

Fields

Modifier and Type	Field and Description
protected double	beta The rate at which weights are demoted; default equal to 1 / LinearThresholdUnit.learningRate.
static double	defaultInitialWeight Default for LinearThresholdUnit.initialWeight.
static double	defaultLearningRate Default for LinearThresholdUnit.learningRate.
static double	defaultThreshold Default for LinearThresholdUnit.threshold.

Fields inherited from class edu.illinois.cs.cogcomp.lbjava.learn.LinearThresholdUnit

allowableValues, bias, defaultThickness, defaultWeightVector, initialWeight, learningRate, negativeThickness, positiveThickness, threshold, weightVector

Fields inherited from class edu.illinois.cs.cogcomp.lbjava.learn.Learner

candidates, encoding, extractor, labeler, labelLexicon, lcFilePath, lexFilePath, lexicon, lossFlag, predictions, readLexiconOnDemand

Fields inherited from class edu.illinois.cs.cogcomp.lbjava.classify.Classifier

containingPackage, name

Constructor Summary

Constructors

Constructor and Description
SparseWinnow() LinearThresholdUnit.learningRate, beta, and LinearThresholdUnit.threshold take default values, while the name of the classifier gets the empty string.
SparseWinnow(double a) Sets LinearThresholdUnit.learningRate to the specified value, beta to 1 / LinearThresholdUnit.learningRate , and the LinearThresholdUnit.threshold takes the default, while the name of the classifier gets the empty string.
SparseWinnow(double a, double b) Sets LinearThresholdUnit.learningRate and beta to the specified values, and the LinearThresholdUnit.threshold takes the default, while the name of the classifier gets the empty string.
SparseWinnow(double a, double b, double t) Sets LinearThresholdUnit.learningRate, beta, and LinearThresholdUnit.threshold to the specified values, while the name of the classifier gets the empty string.
SparseWinnow(double a, double b, double t, double pt) Use this constructor to fit a thick separator, where both the positive and negative sides of the hyperplane will be given the specified thickness, while the name of the classifier gets the empty string.
SparseWinnow(double a, double b, double t, double pt, double nt) Use this constructor to fit a thick separator, where the positive and negative sides of the hyperplane will be given the specified separate thicknesses, while the name of the classifier gets the empty string.
SparseWinnow(double a, double b, double t, double pt, double nt, SparseWeightVector v) Use this constructor to specify an alternative subclass of SparseWeightVector, while the name of the classifier gets the empty string.
SparseWinnow(SparseWinnow.Parameters p) Initializing constructor.
SparseWinnow(String n) LinearThresholdUnit.learningRate, beta, and LinearThresholdUnit.threshold take default values.
SparseWinnow(String n, double a) Sets LinearThresholdUnit.learningRate to the specified value, beta to 1 / LinearThresholdUnit.learningRate , and the LinearThresholdUnit.threshold takes the default.
SparseWinnow(String n, double a, double b) Sets LinearThresholdUnit.learningRate and beta to the specified values, and the LinearThresholdUnit.threshold takes the default.
SparseWinnow(String n, double a, double b, double t) Sets LinearThresholdUnit.learningRate, beta, and LinearThresholdUnit.threshold to the specified values.
SparseWinnow(String n, double a, double b, double t, double pt) Use this constructor to fit a thick separator, where both the positive and negative sides of the hyperplane will be given the specified thickness.
SparseWinnow(String n, double a, double b, double t, double pt, double nt) Use this constructor to fit a thick separator, where the positive and negative sides of the hyperplane will be given the specified separate thicknesses.
SparseWinnow(String n, double a, double b, double t, double pt, double nt, SparseWeightVector v) Use this constructor to specify an alternative subclass of SparseWeightVector.
SparseWinnow(String n, SparseWinnow.Parameters p) Initializing constructor.

Method Summary

Modifier and Type	Method and Description
double	computeLearningRate(int[] exampleFeatures, double[] exampleValues, double s, boolean label) Returns the learning rate, which is LinearThresholdUnit.learningRate (alpha) if it is a positive example, and beta if it is a negative example.
void	demote(int[] exampleFeatures, double[] exampleValues, double rate) Demotion is simply w_i *= betax_i.
double	getBeta() Returns the current value of the beta variable.
double	getLearningRate() Returns the current value of the LinearThresholdUnit.learningRate variable.
Learner.Parameters	getParameters() Retrieves the parameters that are set in this learner.
void	promote(int[] exampleFeatures, double[] exampleValues, double rate) Promotion is simply w_i *= learningRatex_i.
void	read(edu.illinois.cs.cogcomp.core.datastructures.vectors.ExceptionlessInputStream in) Reads the binary representation of a learner with this object's run-time type, overwriting any and all learned or manually specified parameters as well as the label lexicon but without modifying the feature lexicon.
void	setBeta(double t) Sets the beta member variable to the specified value.
void	setLearningRate(double t) Sets the LinearThresholdUnit.learningRate member variable to the specified value.
void	setParameters(SparseWinnow.Parameters p) Sets the values of parameters that control the behavior of this learning algorithm.
void	update(int[] exampleFeatures, double[] exampleValues, double base) This method performs an update w_i *= basex_i, initalizing weights in the weight vector as needed.
void	write(edu.illinois.cs.cogcomp.core.datastructures.vectors.ExceptionlessOutputStream out) Writes the learned function's internal representation in binary form.
void	write(PrintStream out) Writes the algorithm's internal representation as text.

Methods inherited from class edu.illinois.cs.cogcomp.lbjava.learn.LinearThresholdUnit

allowableValues, classify, clone, discreteValue, featureValue, forget, getAllowableValues, getBias, getInitialWeight, getNegativeThickness, getPositiveThickness, getThreshold, getWeightVector, initialize, learn, score, score, scores, setInitialWeight, setLabeler, setNegativeThickness, setParameters, setPositiveThickness, setThickness, setThreshold, shouldDemote, shouldPromote

Methods inherited from class edu.illinois.cs.cogcomp.lbjava.learn.Learner

classify, classify, classify, classify, countFeatures, createPrediction, createPrediction, demandLexicon, discreteValue, discreteValue, doneLearning, doneWithRound, emptyClone, featureValue, featureValue, getCurrentLexicon, getExampleArray, getExampleArray, getExtractor, getLabeler, getLabelLexicon, getLexicon, getLexiconDiscardCounts, getLexiconLocation, getModelLocation, getPrunedLexiconSize, learn, learn, learn, learn, read, readLabelLexicon, readLearner, readLearner, readLearner, readLearner, readLearner, readLearner, readLexicon, readLexicon, readLexiconOnDemand, readLexiconOnDemand, readModel, readModel, readParameters, realValue, realValue, realValue, save, saveLexicon, saveModel, scores, scores, scoresAugmented, setCandidates, setEncoding, setExtractor, setLabelLexicon, setLexicon, setLexiconLocation, setLexiconLocation, setLossFlag, setModelLocation, setModelLocation, setParameters, setReadLexiconOnDemand, unclone, unsetLossFlag, write, writeLexicon, writeModel, writeParameters

Methods inherited from class edu.illinois.cs.cogcomp.lbjava.classify.Classifier

classify, discreteValueArray, getCompositeChildren, getInputType, getOutputType, realValueArray, test, toString, valueIndexOf

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

defaultLearningRate

public static final double defaultLearningRate

Default for LinearThresholdUnit.learningRate.

See Also:

Constant Field Values

defaultThreshold

public static final double defaultThreshold

Default for LinearThresholdUnit.threshold.

See Also:

Constant Field Values

defaultInitialWeight

public static final double defaultInitialWeight

Default for LinearThresholdUnit.initialWeight.

See Also:

Constant Field Values

beta

protected double beta

The rate at which weights are demoted; default equal to 1 / LinearThresholdUnit.learningRate.

Constructor Detail

SparseWinnow

public SparseWinnow()

LinearThresholdUnit.learningRate, beta, and LinearThresholdUnit.threshold take default values, while the name of the classifier gets the empty string.

SparseWinnow

public SparseWinnow(double a)

Sets LinearThresholdUnit.learningRate to the specified value, beta to 1 / LinearThresholdUnit.learningRate , and the LinearThresholdUnit.threshold takes the default, while the name of the classifier gets the empty string.

Parameters:

a - The desired value of the promotion parameter.

SparseWinnow

public SparseWinnow(double a,
                    double b)

Sets LinearThresholdUnit.learningRate and beta to the specified values, and the LinearThresholdUnit.threshold takes the default, while the name of the classifier gets the empty string.

Parameters:

a - The desired value of the promotion parameter.

b - The desired value of the demotion parameter.

SparseWinnow

public SparseWinnow(double a,
                    double b,
                    double t)

Sets LinearThresholdUnit.learningRate, beta, and LinearThresholdUnit.threshold to the specified values, while the name of the classifier gets the empty string.

Parameters:

a - The desired value of the promotion parameter.

b - The desired value of the demotion parameter.

t - The desired threshold value.

SparseWinnow

public SparseWinnow(double a,
                    double b,
                    double t,
                    double pt)

Use this constructor to fit a thick separator, where both the positive and negative sides of the hyperplane will be given the specified thickness, while the name of the classifier gets the empty string.

Parameters:

a - The desired value of the promotion parameter.

b - The desired value of the demotion parameter.

t - The desired threshold value.

pt - The desired positive thickness.

SparseWinnow

public SparseWinnow(double a,
                    double b,
                    double t,
                    double pt,
                    double nt)

Use this constructor to fit a thick separator, where the positive and negative sides of the hyperplane will be given the specified separate thicknesses, while the name of the classifier gets the empty string.

Parameters:

a - The desired value of the promotion parameter.

b - The desired value of the demotion parameter.

t - The desired threshold value.

pt - The desired positive thickness.

nt - The desired negative thickness.

SparseWinnow

public SparseWinnow(double a,
                    double b,
                    double t,
                    double pt,
                    double nt,
                    SparseWeightVector v)

Use this constructor to specify an alternative subclass of SparseWeightVector, while the name of the classifier gets the empty string.

Parameters:

a - The desired value of the promotion parameter.

b - The desired value of the demotion parameter.

t - The desired threshold value.

pt - The desired positive thickness.

nt - The desired negative thickness.

v - An empty sparse weight vector.

SparseWinnow

public SparseWinnow(SparseWinnow.Parameters p)

Initializing constructor. Sets all member variables to their associated settings in the SparseWinnow.Parameters object.

Parameters:

p - The settings of all parameters.

SparseWinnow

public SparseWinnow(String n)

LinearThresholdUnit.learningRate, beta, and LinearThresholdUnit.threshold take default values.

Parameters:

n - The name of the classifier.

SparseWinnow

public SparseWinnow(String n,
                    double a)

Sets LinearThresholdUnit.learningRate to the specified value, beta to 1 / LinearThresholdUnit.learningRate , and the LinearThresholdUnit.threshold takes the default.

Parameters:

n - The name of the classifier.

a - The desired value of the promotion parameter.

SparseWinnow

public SparseWinnow(String n,
                    double a,
                    double b)

Sets LinearThresholdUnit.learningRate and beta to the specified values, and the LinearThresholdUnit.threshold takes the default.

Parameters:

n - The name of the classifier.

a - The desired value of the promotion parameter.

b - The desired value of the demotion parameter.

SparseWinnow

public SparseWinnow(String n,
                    double a,
                    double b,
                    double t)

Sets LinearThresholdUnit.learningRate, beta, and LinearThresholdUnit.threshold to the specified values.

Parameters:

n - The name of the classifier.

a - The desired value of the promotion parameter.

b - The desired value of the demotion parameter.

t - The desired threshold value.

SparseWinnow

public SparseWinnow(String n,
                    double a,
                    double b,
                    double t,
                    double pt)

Use this constructor to fit a thick separator, where both the positive and negative sides of the hyperplane will be given the specified thickness.

Parameters:

n - The name of the classifier.

a - The desired value of the promotion parameter.

b - The desired value of the demotion parameter.

t - The desired threshold value.

pt - The desired positive thickness.

SparseWinnow

public SparseWinnow(String n,
                    double a,
                    double b,
                    double t,
                    double pt,
                    double nt)

Use this constructor to fit a thick separator, where the positive and negative sides of the hyperplane will be given the specified separate thicknesses.

Parameters:

n - The name of the classifier.

a - The desired value of the promotion parameter.

b - The desired value of the demotion parameter.

t - The desired threshold value.

pt - The desired positive thickness.

nt - The desired negative thickness.

SparseWinnow

public SparseWinnow(String n,
                    double a,
                    double b,
                    double t,
                    double pt,
                    double nt,
                    SparseWeightVector v)

Use this constructor to specify an alternative subclass of SparseWeightVector.

Parameters:

n - The name of the classifier.

a - The desired value of the promotion parameter.

b - The desired value of the demotion parameter.

t - The desired threshold value.

pt - The desired positive thickness.

nt - The desired negative thickness.

v - An empty sparse weight vector.

SparseWinnow

public SparseWinnow(String n,
                    SparseWinnow.Parameters p)

Initializing constructor. Sets all member variables to their associated settings in the SparseWinnow.Parameters object.

Parameters:

n - The name of the classifier.

p - The settings of all parameters.

Method Detail

setParameters

public void setParameters(SparseWinnow.Parameters p)

Sets the values of parameters that control the behavior of this learning algorithm.

Parameters:

p - The parameters.

getParameters

public Learner.Parameters getParameters()

Retrieves the parameters that are set in this learner.

Overrides:

getParameters in class LinearThresholdUnit

Returns:

An object containing all the values of the parameters that control the behavior of this learning algorithm.

getLearningRate

public double getLearningRate()

Returns the current value of the LinearThresholdUnit.learningRate variable.

Returns:

The value of the LinearThresholdUnit.learningRate variable.

setLearningRate

public void setLearningRate(double t)

Sets the LinearThresholdUnit.learningRate member variable to the specified value.

Parameters:

t - The new value for LinearThresholdUnit.learningRate.

getBeta

public double getBeta()

Returns the current value of the beta variable.

Returns:

The value of the beta variable.

setBeta

public void setBeta(double t)

Sets the beta member variable to the specified value.

Parameters:

t - The new value for beta.

computeLearningRate

public double computeLearningRate(int[] exampleFeatures,
                                  double[] exampleValues,
                                  double s,
                                  boolean label)

Returns the learning rate, which is LinearThresholdUnit.learningRate (alpha) if it is a positive example, and beta if it is a negative example.

Overrides:

computeLearningRate in class LinearThresholdUnit

Parameters:

exampleFeatures - The example's array of feature indices.

exampleValues - The example's array of feature values.

s - The score.

label - The example label.

Returns:

The appropriate learning rate.

promote

public void promote(int[] exampleFeatures,
                    double[] exampleValues,
                    double rate)

Promotion is simply w_i *= learningRatex_i.

Specified by:

promote in class LinearThresholdUnit

Parameters:

exampleFeatures - The example's array of feature indices.

exampleValues - The example's array of feature values.

rate - The learning rate at which the weights are updated.

demote

public void demote(int[] exampleFeatures,
                   double[] exampleValues,
                   double rate)

Demotion is simply w_i *= betax_i.

Specified by:

demote in class LinearThresholdUnit

Parameters:

exampleFeatures - The example's array of feature indices.

exampleValues - The example's array of feature values.

rate - The learning rate at which the weights are updated.

update

public void update(int[] exampleFeatures,
                   double[] exampleValues,
                   double base)

This method performs an update w_i *= basex_i, initalizing weights in the weight vector as needed.

Parameters:

exampleFeatures - The example's array of feature indices.

exampleValues - The example's array of values.

base - As described above.

write

public void write(PrintStream out)

Writes the algorithm's internal representation as text. In the first line of output, the name of the classifier is printed, followed by LinearThresholdUnit.learningRate, beta, LinearThresholdUnit.initialWeight, LinearThresholdUnit.threshold, LinearThresholdUnit.positiveThickness, LinearThresholdUnit.negativeThickness, and finally LinearThresholdUnit.bias.

Specified by:

write in class Learner

Parameters:

out - The output stream.

write

public void write(edu.illinois.cs.cogcomp.core.datastructures.vectors.ExceptionlessOutputStream out)

Writes the learned function's internal representation in binary form.

Overrides:

write in class LinearThresholdUnit

Parameters:

out - The output stream.

read

public void read(edu.illinois.cs.cogcomp.core.datastructures.vectors.ExceptionlessInputStream in)

Reads the binary representation of a learner with this object's run-time type, overwriting any and all learned or manually specified parameters as well as the label lexicon but without modifying the feature lexicon.

Overrides:

read in class LinearThresholdUnit

Parameters:

in - The input stream.