edu.illinois.cs.cogcomp.lbjava.learn

Class StochasticGradientDescent

java.lang.Object

edu.illinois.cs.cogcomp.lbjava.classify.Classifier

edu.illinois.cs.cogcomp.lbjava.learn.Learner

edu.illinois.cs.cogcomp.lbjava.learn.StochasticGradientDescent

All Implemented Interfaces:

Serializable, Cloneable

public class StochasticGradientDescent
extends Learner

Gradient descent is a batch learning algorithm for function approximation in which the learner tries to follow the gradient of the error function to the solution of minimal error. This implementation is a stochastic approximation to gradient descent in which the approximated function is assumed to have linear form.

This algorithm's user-configurable parameters are stored in member fields of this class. They may be set via either a constructor that names each parameter explicitly or a constructor that takes an instance of Parameters as input. The documentation in each member field in this class indicates the default value of the associated parameter when using the former type of constructor. The documentation of the associated member field in the Parameters class indicates the default value of the parameter when using the latter type of constructor.

Author:

Nick Rizzolo

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	StochasticGradientDescent.Parameters Simply a container for all of StochasticGradientDescent's configurable parameters.

Field Summary

Fields

Modifier and Type	Field and Description
protected double	bias The bias is stored here rather than as an element of the weight vector.
static double	defaultLearningRate Default value for learningRate.
static SparseWeightVector	defaultWeightVector Default for weightVector.
protected double	learningRate The rate at which weights are updated; default defaultLearningRate.
protected SparseWeightVector	weightVector The hypothesis vector; default defaultWeightVector.

Fields inherited from class edu.illinois.cs.cogcomp.lbjava.learn.Learner

candidates, encoding, extractor, labeler, labelLexicon, lcFilePath, lexFilePath, lexicon, lossFlag, predictions, readLexiconOnDemand

Fields inherited from class edu.illinois.cs.cogcomp.lbjava.classify.Classifier

containingPackage, name

Constructor Summary

Constructors

Constructor and Description
StochasticGradientDescent() The learning rate takes the default value, while the name of the classifier gets the empty string.
StochasticGradientDescent(double r) Sets the learning rate to the specified value, while the name of the classifier gets the empty string.
StochasticGradientDescent(StochasticGradientDescent.Parameters p) Initializing constructor.
StochasticGradientDescent(String n) The learning rate takes the default value.
StochasticGradientDescent(String n, double r) Use this constructor to specify an alternative subclass of SparseWeightVector.
StochasticGradientDescent(String n, StochasticGradientDescent.Parameters p) Initializing constructor.

Method Summary

Modifier and Type	Method and Description
FeatureVector	classify(int[] exampleFeatures, double[] exampleValues) Simply computes the dot product of the weight vector and the feature vector extracted from the example object.
Object	clone() Returns a deep clone of this learning algorithm.
Feature	featureValue(int[] f, double[] v) Returns the classification of the given example as a single feature instead of a FeatureVector.
void	forget() Resets the weight vector to all zeros.
double	getLearningRate() Returns the current value of the learningRate variable.
String	getOutputType() Returns a string describing the output feature type of this classifier.
Learner.Parameters	getParameters() Retrieves the parameters that are set in this learner.
void	learn(int[] exampleFeatures, double[] exampleValues, int[] exampleLabels, double[] labelValues) Trains the learning algorithm given an object as an example.
void	read(edu.illinois.cs.cogcomp.core.datastructures.vectors.ExceptionlessInputStream in) Reads the binary representation of a learner with this object's run-time type, overwriting any and all learned or manually specified parameters as well as the label lexicon but without modifying the feature lexicon.
double	realValue(int[] exampleFeatures, double[] exampleValues) Simply computes the dot product of the weight vector and the example
ScoreSet	scores(int[] exampleFeatures, double[] exampleValues) Since this algorithm returns a real feature, it does not return scores.
void	setLearningRate(double t) Sets the learningRate member variable to the specified value.
void	setParameters(StochasticGradientDescent.Parameters p) Sets the values of parameters that control the behavior of this learning algorithm.
void	write(edu.illinois.cs.cogcomp.core.datastructures.vectors.ExceptionlessOutputStream out) Writes the learned function's internal representation in binary form.
void	write(PrintStream out) Writes the algorithm's internal representation as text.

Methods inherited from class edu.illinois.cs.cogcomp.lbjava.learn.Learner

classify, classify, classify, classify, countFeatures, createPrediction, createPrediction, demandLexicon, discreteValue, discreteValue, discreteValue, doneLearning, doneWithRound, emptyClone, featureValue, featureValue, getCurrentLexicon, getExampleArray, getExampleArray, getExtractor, getLabeler, getLabelLexicon, getLexicon, getLexiconDiscardCounts, getLexiconLocation, getModelLocation, getPrunedLexiconSize, initialize, learn, learn, learn, learn, read, readLabelLexicon, readLearner, readLearner, readLearner, readLearner, readLearner, readLearner, readLexicon, readLexicon, readLexiconOnDemand, readLexiconOnDemand, readModel, readModel, readParameters, realValue, realValue, save, saveLexicon, saveModel, scores, scores, scoresAugmented, setCandidates, setEncoding, setExtractor, setLabeler, setLabelLexicon, setLexicon, setLexiconLocation, setLexiconLocation, setLossFlag, setModelLocation, setModelLocation, setParameters, setReadLexiconOnDemand, unclone, unsetLossFlag, write, writeLexicon, writeModel, writeParameters

Methods inherited from class edu.illinois.cs.cogcomp.lbjava.classify.Classifier

allowableValues, classify, discreteValueArray, getCompositeChildren, getInputType, realValueArray, test, toString, valueIndexOf

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

defaultLearningRate

public static final double defaultLearningRate

Default value for learningRate.

See Also:

Constant Field Values

defaultWeightVector

public static final SparseWeightVector defaultWeightVector

Default for weightVector.

weightVector

protected SparseWeightVector weightVector

The hypothesis vector; default defaultWeightVector.

bias

protected double bias

The bias is stored here rather than as an element of the weight vector.

learningRate

protected double learningRate

The rate at which weights are updated; default defaultLearningRate.

Constructor Detail

StochasticGradientDescent

public StochasticGradientDescent()

The learning rate takes the default value, while the name of the classifier gets the empty string.

StochasticGradientDescent

public StochasticGradientDescent(double r)

Sets the learning rate to the specified value, while the name of the classifier gets the empty string.

Parameters:

r - The desired learning rate value.

StochasticGradientDescent

public StochasticGradientDescent(StochasticGradientDescent.Parameters p)

Initializing constructor. Sets all member variables to their associated settings in the StochasticGradientDescent.Parameters object.

Parameters:

p - The settings of all parameters.

StochasticGradientDescent

public StochasticGradientDescent(String n)

The learning rate takes the default value.

Parameters:

n - The name of the classifier.

StochasticGradientDescent

public StochasticGradientDescent(String n,
                                 double r)

Use this constructor to specify an alternative subclass of SparseWeightVector.

Parameters:

n - The name of the classifier.

r - The desired learning rate value.

StochasticGradientDescent

public StochasticGradientDescent(String n,
                                 StochasticGradientDescent.Parameters p)

Initializing constructor. Sets all member variables to their associated settings in the StochasticGradientDescent.Parameters object.

Parameters:

n - The name of the classifier.

p - The settings of all parameters.

Method Detail

setParameters

public void setParameters(StochasticGradientDescent.Parameters p)

Sets the values of parameters that control the behavior of this learning algorithm.

Parameters:

p - The parameters.

getParameters

public Learner.Parameters getParameters()

Retrieves the parameters that are set in this learner.

Overrides:

getParameters in class Learner

Returns:

An object containing all the values of the parameters that control the behavior of this learning algorithm.

getLearningRate

public double getLearningRate()

Returns the current value of the learningRate variable.

Returns:

The value of the learningRate variable.

setLearningRate

public void setLearningRate(double t)

Sets the learningRate member variable to the specified value.

Parameters:

t - The new value for learningRate.

forget

public void forget()

Resets the weight vector to all zeros.

Overrides:

forget in class Learner

getOutputType

public String getOutputType()

Returns a string describing the output feature type of this classifier.

Overrides:

getOutputType in class Classifier

Returns:

"real"

learn

public void learn(int[] exampleFeatures,
                  double[] exampleValues,
                  int[] exampleLabels,
                  double[] labelValues)

Trains the learning algorithm given an object as an example.

Specified by:

learn in class Learner

Parameters:

exampleFeatures - The example's array of feature indices.

exampleValues - The example's array of feature values.

exampleLabels - The example's label(s).

labelValues - The labels' values.

scores

public ScoreSet scores(int[] exampleFeatures,
                       double[] exampleValues)

Since this algorithm returns a real feature, it does not return scores.

Specified by:

scores in class Learner

Parameters:

exampleFeatures - The example's array of feature indices.

exampleValues - The example's array of feature values.

Returns:

null

featureValue

public Feature featureValue(int[] f,
                            double[] v)

Returns the classification of the given example as a single feature instead of a FeatureVector.

Overrides:

featureValue in class Learner

Parameters:

f - The features array.

v - The values array.

Returns:

The classification of the example as a feature.

realValue

public double realValue(int[] exampleFeatures,
                        double[] exampleValues)

Simply computes the dot product of the weight vector and the example

Overrides:

realValue in class Learner

Parameters:

exampleFeatures - The example's array of feature indices.

exampleValues - The example's array of feature values.

Returns:

The computed real value.

classify

public FeatureVector classify(int[] exampleFeatures,
                              double[] exampleValues)

Simply computes the dot product of the weight vector and the feature vector extracted from the example object.

Specified by:

classify in class Learner

Parameters:

exampleFeatures - The example's array of feature indices.

exampleValues - The example's array of feature values.

Returns:

The computed feature (in a vector).

write

public void write(PrintStream out)

Writes the algorithm's internal representation as text. In the first line of output, the name of the classifier is printed, followed by learningRate and bias.

Specified by:

write in class Learner

Parameters:

out - The output stream.

write

public void write(edu.illinois.cs.cogcomp.core.datastructures.vectors.ExceptionlessOutputStream out)

Writes the learned function's internal representation in binary form.

Overrides:

write in class Learner

Parameters:

out - The output stream.

read

public void read(edu.illinois.cs.cogcomp.core.datastructures.vectors.ExceptionlessInputStream in)

Reads the binary representation of a learner with this object's run-time type, overwriting any and all learned or manually specified parameters as well as the label lexicon but without modifying the feature lexicon.

Overrides:

read in class Learner

Parameters:

in - The input stream.

clone

public Object clone()

Returns a deep clone of this learning algorithm.

Overrides:

clone in class Learner

Returns:

A shallow clone.