LBJ2.learn
Class SparseAveragedPerceptron

java.lang.Object
  LBJ2.classify.Classifier
      LBJ2.learn.Learner
          LBJ2.learn.LinearThresholdUnit
              LBJ2.learn.SparsePerceptron
                  LBJ2.learn.SparseAveragedPerceptron

All Implemented Interfaces:

java.io.Serializable, java.lang.Cloneable

public class SparseAveragedPerceptron
extends SparsePerceptron

An approximation to voted Perceptron, in which a weighted average of the weight vectors arrived at during training becomes the weight vector used to make predictions after training.

During training, after each example e_i is processed, the weight vector w_i becomes the active weight vector used to make predictions on future training examples. If a mistake was made on e_i, w_i will be different than w_{i - 1}. Otherwise, it will remain unchanged.

After training, each distinct weight vector arrived at during training is associated with an integer weight equal to the number of examples whose training made that weight vector active. A new weight vector w^* is computed by taking the average of all these weight vectors weighted as described. w^* is used to make all predictions returned to the user through methods such as Classifier.classify(Object) or Classifier.discreteValue(Object).

The above description is a useful way to think about the operation of this Learner. However, the user should note that this implementation never explicitly stores w^*. Instead, it is computed efficiently on demand. Thus, interspersed online training and evaluation is efficient and operates as expected.

It is assumed that Learner.labeler is a single discrete classifier that produces the same feature for every example object and that the values that feature may take are available through the Classifier.allowableValues() method. The second value returned from Classifier.allowableValues() is treated as "positive", and it is assumed there are exactly 2 allowable values. Assertions will produce error messages if these assumptions do not hold.

See Also:

Serialized Form

Nested Class Summary

static class	SparseAveragedPerceptron.AveragedWeightVector This implementation of a sparse weight vector associates two doubles with each Feature.
static class	SparseAveragedPerceptron.Parameters Simply a container for all of SparseAveragedPerceptron's configurable parameters.

Field Summary

protected double	averagedBias Keeps the extra information necessary to compute the averaged bias.
static SparseAveragedPerceptron.AveragedWeightVector	defaultWeightVector Default for LinearThresholdUnit.weightVector.

Fields inherited from class LBJ2.learn.SparsePerceptron

defaultLearningRate, learningRate

Fields inherited from class LBJ2.learn.LinearThresholdUnit

allowableValues, bias, defaultInitialWeight, defaultThickness, defaultThreshold, initialWeight, negativeThickness, positiveThickness, threshold, weightVector

Fields inherited from class LBJ2.learn.Learner

extractor, labeler

Fields inherited from class LBJ2.classify.Classifier

containingPackage, name

Constructor Summary

SparseAveragedPerceptron()
The learning rate and threshold take default values, while the name of the classifier gets the empty string.

SparseAveragedPerceptron(double r)
Sets the learning rate to the specified value, and the threshold takes the default, while the name of the classifier gets the empty string.

SparseAveragedPerceptron(double r, double t)
Sets the learning rate and threshold to the specified values, while the name of the classifier gets the empty string.

SparseAveragedPerceptron(double r, double t, double pt)
Use this constructor to fit a thick separator, where both the positive and negative sides of the hyperplane will be given the specified thickness, while the name of the classifier gets the empty string.

SparseAveragedPerceptron(double r, double t, double pt, double nt)
Use this constructor to fit a thick separator, where the positive and negative sides of the hyperplane will be given the specified separate thicknesses, while the name of the classifier gets the empty string.

SparseAveragedPerceptron(SparseAveragedPerceptron.Parameters p)
Initializing constructor.

SparseAveragedPerceptron(java.lang.String n)
The learning rate and threshold take default values.

SparseAveragedPerceptron(java.lang.String n, double r)
Sets the learning rate to the specified value, and the threshold takes the default.

SparseAveragedPerceptron(java.lang.String n, double r, double t)
Sets the learning rate and threshold to the specified values.

SparseAveragedPerceptron(java.lang.String n, double r, double t, double pt)
Use this constructor to fit a thick separator, where both the positive and negative sides of the hyperplane will be given the specified thickness.

SparseAveragedPerceptron(java.lang.String n, double r, double t, double pt, double nt)
Use this constructor to fit a thick separator, where the positive and negative sides of the hyperplane will be given the specified separate thicknesses.

SparseAveragedPerceptron(java.lang.String n, SparseAveragedPerceptron.Parameters p)
Initializing constructor.

Method Summary

void	demote(java.lang.Object example) Scales the feature vector produced by the extractor by the learning rate and subtracts it from the weight vector.
void	forget() Resets the weight vector to all zeros.
void	learn(java.lang.Object example) This method works just like LinearThresholdUnit.learn(Object), except it assumes that LinearThresholdUnit.weightVector references a SparseAveragedPerceptron.AveragedWeightVector and it notifies that vector when it got an example correct in addition to updating it when it makes a mistake.
void	promote(java.lang.Object example) Scales the feature vector produced by the extractor by the learning rate and adds it to the weight vector.
double	score(java.lang.Object example) The score of the specified object is equal to w * x + bias where * is dot product, w is the weight vector, and x is the feature vector produced by the extractor.
void	write(java.io.PrintStream out) Writes the algorithm's internal representation as text.

Methods inherited from class LBJ2.learn.SparsePerceptron

clone, getLearningRate, setLearningRate

Methods inherited from class LBJ2.learn.LinearThresholdUnit

allowableValues, classify, getInitialWeight, getNegativeThickness, getPositiveThickness, getThreshold, scores, setInitialWeight, setLabeler, setNegativeThickness, setPositiveThickness, setThickness, setThreshold

Methods inherited from class LBJ2.learn.Learner

doneLearning, getExtractor, getLabeler, learn, save, setExtractor

Methods inherited from class LBJ2.classify.Classifier

binaryRead, binaryRead, binaryRead, binaryRead, binaryWrite, binaryWrite, classify, discreteValue, discreteValueArray, getCompositeChildren, getInputType, getOutputType, realValue, realValueArray, test, toString, valueIndexOf

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

defaultWeightVector

public static final SparseAveragedPerceptron.AveragedWeightVector defaultWeightVector

Default for LinearThresholdUnit.weightVector.

averagedBias

protected double averagedBias

Keeps the extra information necessary to compute the averaged bias.

Constructor Detail

SparseAveragedPerceptron

public SparseAveragedPerceptron()

The learning rate and threshold take default values, while the name of the classifier gets the empty string.

SparseAveragedPerceptron

public SparseAveragedPerceptron(double r)

Sets the learning rate to the specified value, and the threshold takes the default, while the name of the classifier gets the empty string.

Parameters:

r - The desired learning rate value.

SparseAveragedPerceptron

public SparseAveragedPerceptron(double r,
                                double t)

Sets the learning rate and threshold to the specified values, while the name of the classifier gets the empty string.

Parameters:

r - The desired learning rate value.

t - The desired threshold value.

SparseAveragedPerceptron

public SparseAveragedPerceptron(double r,
                                double t,
                                double pt)

Use this constructor to fit a thick separator, where both the positive and negative sides of the hyperplane will be given the specified thickness, while the name of the classifier gets the empty string.

Parameters:

r - The desired learning rate value.

t - The desired threshold value.

pt - The desired thickness.

SparseAveragedPerceptron

public SparseAveragedPerceptron(double r,
                                double t,
                                double pt,
                                double nt)

Use this constructor to fit a thick separator, where the positive and negative sides of the hyperplane will be given the specified separate thicknesses, while the name of the classifier gets the empty string.

Parameters:

r - The desired learning rate value.

t - The desired threshold value.

pt - The desired positive thickness.

nt - The desired negative thickness.

SparseAveragedPerceptron

public SparseAveragedPerceptron(SparseAveragedPerceptron.Parameters p)

Initializing constructor. Sets all member variables to their associated settings in the SparseAveragedPerceptron.Parameters object.

Parameters:

p - The settings of all parameters.

SparseAveragedPerceptron

public SparseAveragedPerceptron(java.lang.String n)

The learning rate and threshold take default values.

Parameters:

n - The name of the classifier.

SparseAveragedPerceptron

public SparseAveragedPerceptron(java.lang.String n,
                                double r)

Sets the learning rate to the specified value, and the threshold takes the default.

Parameters:

n - The name of the classifier.

r - The desired learning rate value.

SparseAveragedPerceptron

public SparseAveragedPerceptron(java.lang.String n,
                                double r,
                                double t)

Sets the learning rate and threshold to the specified values.

Parameters:

n - The name of the classifier.

r - The desired learning rate value.

t - The desired threshold value.

SparseAveragedPerceptron

public SparseAveragedPerceptron(java.lang.String n,
                                double r,
                                double t,
                                double pt)

Use this constructor to fit a thick separator, where both the positive and negative sides of the hyperplane will be given the specified thickness.

Parameters:

n - The name of the classifier.

r - The desired learning rate value.

t - The desired threshold value.

pt - The desired thickness.

SparseAveragedPerceptron

public SparseAveragedPerceptron(java.lang.String n,
                                double r,
                                double t,
                                double pt,
                                double nt)

Use this constructor to fit a thick separator, where the positive and negative sides of the hyperplane will be given the specified separate thicknesses.

Parameters:

n - The name of the classifier.

r - The desired learning rate value.

t - The desired threshold value.

pt - The desired positive thickness.

nt - The desired negative thickness.

SparseAveragedPerceptron

public SparseAveragedPerceptron(java.lang.String n,
                                SparseAveragedPerceptron.Parameters p)

Initializing constructor. Sets all member variables to their associated settings in the SparseAveragedPerceptron.Parameters object.

Parameters:

n - The name of the classifier.

p - The settings of all parameters.

Method Detail

score

public double score(java.lang.Object example)

The score of the specified object is equal to w * x + bias where * is dot product, w is the weight vector, and x is the feature vector produced by the extractor.

Overrides:

score in class LinearThresholdUnit

Parameters:

example - The example object.

Returns:

The result of the dot product plus the bias.

promote

public void promote(java.lang.Object example)

Scales the feature vector produced by the extractor by the learning rate and adds it to the weight vector.

Overrides:

promote in class SparsePerceptron

Parameters:

example - The example object.

demote

public void demote(java.lang.Object example)

Scales the feature vector produced by the extractor by the learning rate and subtracts it from the weight vector.

Overrides:

demote in class SparsePerceptron

Parameters:

example - The example object.

learn

public void learn(java.lang.Object example)

This method works just like LinearThresholdUnit.learn(Object), except it assumes that LinearThresholdUnit.weightVector references a SparseAveragedPerceptron.AveragedWeightVector and it notifies that vector when it got an example correct in addition to updating it when it makes a mistake.

Overrides:

learn in class LinearThresholdUnit

Parameters:

example - The example object.

forget

public void forget()

Resets the weight vector to all zeros.

Overrides:

forget in class LinearThresholdUnit

write

public void write(java.io.PrintStream out)

Writes the algorithm's internal representation as text. In the first line of output, the name of the classifier is printed, followed by SparsePerceptron.learningRate, LinearThresholdUnit.initialWeight, LinearThresholdUnit.threshold, LinearThresholdUnit.positiveThickness, LinearThresholdUnit.negativeThickness, LinearThresholdUnit.bias, and finally averagedBias.

Overrides:

write in class SparsePerceptron

Parameters:

out - The output stream.