edu.illinois.cs.cogcomp.bigdata.lucene

Class WikiURLAnalyzer

java.lang.Object

org.apache.lucene.analysis.Analyzer

edu.illinois.cs.cogcomp.bigdata.lucene.WikiURLAnalyzer

All Implemented Interfaces:

Closeable, AutoCloseable

public class WikiURLAnalyzer
extends org.apache.lucene.analysis.Analyzer

Only use this class for short strings

Author:

cheng88

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	WikiURLAnalyzer.URLField

Nested classes/interfaces inherited from class org.apache.lucene.analysis.Analyzer

org.apache.lucene.analysis.Analyzer.ReuseStrategy, org.apache.lucene.analysis.Analyzer.TokenStreamComponents

Field Summary

Fields

Modifier and Type	Field and Description
static String	normalizeChars
static String	replacement

Fields inherited from class org.apache.lucene.analysis.Analyzer

GLOBAL_REUSE_STRATEGY, PER_FIELD_REUSE_STRATEGY

Constructor Summary

Constructors

Constructor and Description
WikiURLAnalyzer()

Method Summary

Modifier and Type	Method and Description
protected org.apache.lucene.analysis.Analyzer.TokenStreamComponents	createComponents(String fieldName)
static List<String>	pruningTokenization(String s)

Methods inherited from class org.apache.lucene.analysis.Analyzer

close, getOffsetGap, getPositionIncrementGap, getReuseStrategy, getVersion, initReader, setVersion, tokenStream, tokenStream

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

normalizeChars

public static final String normalizeChars

See Also:

Constant Field Values

replacement

public static final String replacement

See Also:

Constant Field Values

Constructor Detail

WikiURLAnalyzer

public WikiURLAnalyzer()

Method Detail

createComponents

protected org.apache.lucene.analysis.Analyzer.TokenStreamComponents createComponents(String fieldName)

Specified by:

createComponents in class org.apache.lucene.analysis.Analyzer

pruningTokenization

public static List<String> pruningTokenization(String s)