edu.illinois.cs.cogcomp.lbjava.learn

Class NaiveBayes

java.lang.Object

edu.illinois.cs.cogcomp.lbjava.classify.Classifier

edu.illinois.cs.cogcomp.lbjava.learn.Learner

edu.illinois.cs.cogcomp.lbjava.learn.NaiveBayes

All Implemented Interfaces:

Serializable, Cloneable

public class NaiveBayes
extends Learner

Naive Bayes is a multi-class learner that uses prediction value counts and feature counts given a particular prediction value to select the most likely prediction value. More precisely, a score s_v for a given prediction value v is computed such that e^s_v is proportional to

P(v) Prod_f P(f|v)

where Prod is a multiplication quantifier over f, and f stands for a feature. The value corresponding to the highest score is selected as the prediction. Feature values that were never observed given a particular prediction value during training are smoothed with a configurable constant that defaults to e^-15.

This Learner learns a discrete classifier from other discrete classifiers. Features coming from real classifiers are ignored. It is also assumed that a single discrete label feature will be produced in association with each example object. A feature taking one of the values observed in that label feature will be produced by the learned classifier.

This algorithm's user-configurable parameters are stored in member fields of this class. They may be set via either a constructor that names each parameter explicitly or a constructor that takes an instance of Parameters as input. The documentation in each member field in this class indicates the default value of the associated parameter when using the former type of constructor. The documentation of the associated member field in the Parameters class indicates the default value of the parameter when using the latter type of constructor.

Author:

Nick Rizzolo

See Also:

NaiveBayes.NaiveBayesVector, Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	NaiveBayes.Count A Count object stores two doubles, one which holds a accumulated count value and the other intended to hold the natural logarithm of the count.
protected class	NaiveBayes.NaiveBayesVector Keeps track of all the counts associated with a given label.
static class	NaiveBayes.Parameters Simply a container for all of NaiveBayes's configurable parameters.

Field Summary

Fields

Modifier and Type	Field and Description
static int	defaultSmoothing The default conditional feature probability is edefaultSmoothing .
protected edu.illinois.cs.cogcomp.core.datastructures.vectors.OVector	network One NaiveBayes.NaiveBayesVector for each observed prediction value.
protected double	smoothing The exponential of this number is used as the conditional probability of a feature that was never observed during training; default defaultSmoothing.

Fields inherited from class edu.illinois.cs.cogcomp.lbjava.learn.Learner

candidates, encoding, extractor, labeler, labelLexicon, lcFilePath, lexFilePath, lexicon, lossFlag, predictions, readLexiconOnDemand

Fields inherited from class edu.illinois.cs.cogcomp.lbjava.classify.Classifier

containingPackage, name

Constructor Summary

Constructors

Constructor and Description
NaiveBayes() Default constructor.
NaiveBayes(double smooth) Initializes the smoothing constant.
NaiveBayes(NaiveBayes.Parameters p) Initializing constructor.
NaiveBayes(String n) Initializes the name of the classifier.
NaiveBayes(String name, double smooth) Initializes the name and smoothing constant.
NaiveBayes(String n, NaiveBayes.Parameters p) Initializing constructor.

Method Summary

Modifier and Type	Method and Description
FeatureVector	classify(int[] exampleFeatures, double[] exampleValues) Prediction value counts and feature counts given a particular prediction value are used to select the most likely prediction value.
Object	clone() Returns a deep clone of this learning algorithm.
String	discreteValue(int[] exampleFeatures, double[] exampleValues) Prediction value counts and feature counts given a particular prediction value are used to select the most likely prediction value.
Feature	featureValue(int[] f, double[] v) Returns the classification of the given example as a single feature instead of a FeatureVector.
void	forget() Clears the network.
Learner.Parameters	getParameters() Retrieves the parameters that are set in this learner.
void	learn(int[] exampleFeatures, double[] exampleValues, int[] exampleLabels, double[] labelValues) Trains the learning algorithm given an object as an example.
void	read(edu.illinois.cs.cogcomp.core.datastructures.vectors.ExceptionlessInputStream in) Reads the binary representation of a learner with this object's run-time type, overwriting any and all learned or manually specified parameters as well as the label lexicon but without modifying the feature lexicon.
ScoreSet	scores(int[] exampleFeatures, double[] exampleValues) The scores in the returned ScoreSet are the posterior probabilities of each possible label given the example.
void	setLabeler(Classifier l) Sets the labeler.
void	setParameters(NaiveBayes.Parameters p) Sets the values of parameters that control the behavior of this learning algorithm.
void	setSmoothing(double s) Sets the smoothing parameter to the specified value.
void	write(edu.illinois.cs.cogcomp.core.datastructures.vectors.ExceptionlessOutputStream out) Writes the learned function's internal representation in binary form.
void	write(PrintStream out) Writes the algorithm's internal representation as text.

Methods inherited from class edu.illinois.cs.cogcomp.lbjava.learn.Learner

classify, classify, classify, classify, countFeatures, createPrediction, createPrediction, demandLexicon, discreteValue, discreteValue, doneLearning, doneWithRound, emptyClone, featureValue, featureValue, getCurrentLexicon, getExampleArray, getExampleArray, getExtractor, getLabeler, getLabelLexicon, getLexicon, getLexiconDiscardCounts, getLexiconLocation, getModelLocation, getPrunedLexiconSize, initialize, learn, learn, learn, learn, read, readLabelLexicon, readLearner, readLearner, readLearner, readLearner, readLearner, readLearner, readLexicon, readLexicon, readLexiconOnDemand, readLexiconOnDemand, readModel, readModel, readParameters, realValue, realValue, realValue, save, saveLexicon, saveModel, scores, scores, scoresAugmented, setCandidates, setEncoding, setExtractor, setLabelLexicon, setLexicon, setLexiconLocation, setLexiconLocation, setLossFlag, setModelLocation, setModelLocation, setParameters, setReadLexiconOnDemand, unclone, unsetLossFlag, write, writeLexicon, writeModel, writeParameters

Methods inherited from class edu.illinois.cs.cogcomp.lbjava.classify.Classifier

allowableValues, classify, discreteValueArray, getCompositeChildren, getInputType, getOutputType, realValueArray, test, toString, valueIndexOf

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

defaultSmoothing

public static final int defaultSmoothing

The default conditional feature probability is e^{defaultSmoothing} .

See Also:

Constant Field Values

smoothing

protected double smoothing

The exponential of this number is used as the conditional probability of a feature that was never observed during training; default defaultSmoothing.

network

protected edu.illinois.cs.cogcomp.core.datastructures.vectors.OVector network

One NaiveBayes.NaiveBayesVector for each observed prediction value.

Constructor Detail

NaiveBayes

public NaiveBayes()

Default constructor.

NaiveBayes

public NaiveBayes(double smooth)

Initializes the smoothing constant.

Parameters:

smooth - The exponential of this number is used as the conditional probability of a feature that was never observed during training.

NaiveBayes

public NaiveBayes(NaiveBayes.Parameters p)

Initializing constructor. Sets all member variables to their associated settings in the NaiveBayes.Parameters object.

Parameters:

p - The settings of all parameters.

NaiveBayes

public NaiveBayes(String n)

Initializes the name of the classifier.

Parameters:

n - The classifier's name.

NaiveBayes

public NaiveBayes(String name,
                  double smooth)

Initializes the name and smoothing constant.

Parameters:

name - The classifier's name.

smooth - The exponential of this number is used as the conditional probability of a feature that was never observed during training.

NaiveBayes

public NaiveBayes(String n,
                  NaiveBayes.Parameters p)

Initializing constructor. Sets all member variables to their associated settings in the NaiveBayes.Parameters object.

Parameters:

n - The name of the classifier.

p - The settings of all parameters.

Method Detail

setParameters

public void setParameters(NaiveBayes.Parameters p)

Sets the values of parameters that control the behavior of this learning algorithm.

Parameters:

p - The parameters.

getParameters

public Learner.Parameters getParameters()

Retrieves the parameters that are set in this learner.

Overrides:

getParameters in class Learner

Returns:

An object containing all the values of the parameters that control the behavior of this learning algorithm.

setSmoothing

public void setSmoothing(double s)

Sets the smoothing parameter to the specified value.

Parameters:

s - The new value for the smoothing parameter.

setLabeler

public void setLabeler(Classifier l)

Sets the labeler.

Overrides:

setLabeler in class Learner

Parameters:

l - A labeling classifier.

learn

public void learn(int[] exampleFeatures,
                  double[] exampleValues,
                  int[] exampleLabels,
                  double[] labelValues)

Trains the learning algorithm given an object as an example.

Specified by:

learn in class Learner

Parameters:

exampleFeatures - The example's array of feature indices.

exampleValues - The example's array of feature values.

exampleLabels - The example's label(s).

labelValues - The labels' values.

forget

public void forget()

Clears the network.

Overrides:

forget in class Learner

scores

public ScoreSet scores(int[] exampleFeatures,
                       double[] exampleValues)

The scores in the returned ScoreSet are the posterior probabilities of each possible label given the example.

Specified by:

scores in class Learner

Parameters:

exampleFeatures - The example's array of feature indices.

exampleValues - The example's array of feature values.

Returns:

A set of scores indicating the degree to which each possible discrete classification value is associated with the given example object.

featureValue

public Feature featureValue(int[] f,
                            double[] v)

Returns the classification of the given example as a single feature instead of a FeatureVector.

Overrides:

featureValue in class Learner

Parameters:

f - The features array.

v - The values array.

Returns:

The classification of the example as a feature.

discreteValue

public String discreteValue(int[] exampleFeatures,
                            double[] exampleValues)

Prediction value counts and feature counts given a particular prediction value are used to select the most likely prediction value.

Overrides:

discreteValue in class Learner

Parameters:

exampleFeatures - The example's array of feature indices.

exampleValues - The example's array of feature values.

Returns:

The most likely discrete value.

classify

public FeatureVector classify(int[] exampleFeatures,
                              double[] exampleValues)

Prediction value counts and feature counts given a particular prediction value are used to select the most likely prediction value.

Specified by:

classify in class Learner

Parameters:

exampleFeatures - The example's array of feature indices.

exampleValues - The example's array of feature values.

Returns:

A single discrete feature, set to the most likely value.

write

public void write(PrintStream out)

Writes the algorithm's internal representation as text.

Specified by:

write in class Learner

Parameters:

out - The output stream.

write

public void write(edu.illinois.cs.cogcomp.core.datastructures.vectors.ExceptionlessOutputStream out)

Writes the learned function's internal representation in binary form.

Overrides:

write in class Learner

Parameters:

out - The output stream.

read

public void read(edu.illinois.cs.cogcomp.core.datastructures.vectors.ExceptionlessInputStream in)

Reads the binary representation of a learner with this object's run-time type, overwriting any and all learned or manually specified parameters as well as the label lexicon but without modifying the feature lexicon.

Overrides:

read in class Learner

Parameters:

in - The input stream.

clone

public Object clone()

Returns a deep clone of this learning algorithm.

Overrides:

clone in class Learner

Returns:

A shallow clone.