edu.illinois.cs.cogcomp.edison.utilities

Class WriteSVMLightFormat

java.lang.Object

edu.illinois.cs.cogcomp.edison.utilities.WriteSVMLightFormat

public class WriteSVMLightFormat
extends Object

Writes Edison Features into SVMLight format for use by non-CCG learning frameworks SVMLight format is: [label] ([featid]:[featvalue])+ -- only active features are represented. -- feature ids must be given in ascending numerical order. WriteSVMLightFormat maintains a Lexicon to track feature ids, and label ids if binary classifier is not specified at construction. Binary classification tasks: expects WriteSVMLightFormat.TRUE_LAB and WriteSVMLightFormat.FALSE_LAB as labels for positive, negative examples respectively.

Author:

mssammon

Field Summary

Fields

Modifier and Type	Field and Description
static String	FALSE_LAB
static String	TRUE_LAB

Constructor Summary

Constructors

Constructor and Description
WriteSVMLightFormat() default constructor: binary task, no bias feature, don't store string values
WriteSVMLightFormat(boolean isBinaryTask, boolean hasBias, boolean storeStringValues)

Method Summary

Modifier and Type	Method and Description
String	getLexiconContentsAsString() return a String representation of the lexicon (tab-separated key/value pairs, where the key is an integer and the value is a feature's text representation
String	writeFeatureExample(String label, Collection<Feature> activeFeatures) Generate a String value corresponding to a SVMLight format example from Edison feature representation (String label and Collection of Feature).
void	writeLexiconToOutputStream(OutputStream out) write the lexicon out to an outputstream.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

TRUE_LAB

public static final String TRUE_LAB

FALSE_LAB

public static final String FALSE_LAB

Constructor Detail

WriteSVMLightFormat

public WriteSVMLightFormat()

default constructor: binary task, no bias feature, don't store string values

WriteSVMLightFormat

public WriteSVMLightFormat(boolean isBinaryTask,
                           boolean hasBias,
                           boolean storeStringValues)

Method Detail

getLexiconContentsAsString

public String getLexiconContentsAsString()
                                  throws IOException

return a String representation of the lexicon (tab-separated key/value pairs, where the key is an integer and the value is a feature's text representation

Returns:

Throws:

IOException

writeLexiconToOutputStream

public void writeLexiconToOutputStream(OutputStream out)
                                throws IOException

write the lexicon out to an outputstream. Note that for non-binary tasks, there may be duplicate entries for integer ids: labels may have the same id as features.

Parameters:

out - where the lexicon will be written

Throws:

IOException

writeFeatureExample

public String writeFeatureExample(String label,
                                  Collection<Feature> activeFeatures)

Generate a String value corresponding to a SVMLight format example from Edison feature representation (String label and Collection of Feature). Stores a mapping between features and integer ids so that when a new example is passed in, if a feature is the same as an example that has already been processed, that feature will get the same integer id as it did previously. From SVMLight documentation, binary classification labels must be {-1,1} for binary problem, but otherwise labels are just integers -- one integer per class for multi-class problem integer rank for ranking problem Apparently, no problem for label to have same integer id as feature value Assumes sparse boolean feature representation (if a feature is active, it has a unique string identifier in the Collection argument). IMPORTANT: feature ids MUST be written out in ascending order.

Parameters:

label -

activeFeatures -

Returns: