Uses of Class
edu.illinois.cs.cogcomp.nlp.tokenizer.Tokenizer.Tokenization

Packages that use Tokenizer.Tokenization

Package	Description
edu.illinois.cs.cogcomp.annotation
edu.illinois.cs.cogcomp.curator
edu.illinois.cs.cogcomp.nlp.tokenizer
edu.illinois.cs.cogcomp.nlp.utility
edu.illinois.cs.cogcomp.tokenizer

Uses of Tokenizer.Tokenization in edu.illinois.cs.cogcomp.annotation

Methods in edu.illinois.cs.cogcomp.annotation with parameters of type Tokenizer.Tokenization

Modifier and Type	Method and Description
TextAnnotation	AnnotatorService.createAnnotatedTextAnnotation(String corpusId, String textId, String text, Tokenizer.Tokenization tokenization) A convenience method for creating a TextAnnotation and adding all the Views supported by this AnnotatorService.
TextAnnotation	BasicAnnotatorService.createAnnotatedTextAnnotation(String corpusId, String textId, String text, Tokenizer.Tokenization tokenization) A convenience method for creating a TextAnnotation and adding all the Views supported by this AnnotatorService.
TextAnnotation	AnnotatorService.createAnnotatedTextAnnotation(String corpusId, String textId, String text, Tokenizer.Tokenization tokenization, Set<String> viewNames) An overloaded version of AnnotatorService.createAnnotatedTextAnnotation(String, String, String, Tokenizer.Tokenization) that adds only the Views requested.
TextAnnotation	BasicAnnotatorService.createAnnotatedTextAnnotation(String corpusId, String textId, String text, Tokenizer.Tokenization tokenization, Set<String> viewNames) An overloaded version of BasicAnnotatorService.createAnnotatedTextAnnotation(String, String, String) that adds only the Views requested.
TextAnnotation	AnnotatorService.createBasicTextAnnotation(String corpusId, String docId, String text, Tokenizer.Tokenization tokenization) A convenience method for creating a TextAnnotation while respecting the pre-tokenization of text passed in the form of Tokenizer.Tokenization.
TextAnnotation	BasicAnnotatorService.createBasicTextAnnotation(String corpusId, String docId, String text, Tokenizer.Tokenization tokenization) A convenience method for creating a TextAnnotation while respecting the pre-tokenization of text passed in the form of Tokenizer.Tokenization.
TextAnnotation	TextAnnotationBuilder.createTextAnnotation(String corpusId, String textId, String text, Tokenizer.Tokenization tokenization) A method for creating TextAnnotation by respecting the pre-tokenization of text passed as an instance of Tokenizer.Tokenization.
TextAnnotation	BasicTextAnnotationBuilder.createTextAnnotation(String corpusId, String textId, String text, Tokenizer.Tokenization tokenization)

Uses of Tokenizer.Tokenization in edu.illinois.cs.cogcomp.curator

Methods in edu.illinois.cs.cogcomp.curator with parameters of type Tokenizer.Tokenization

Modifier and Type	Method and Description
TextAnnotation	CuratorAnnotatorService.createAnnotatedTextAnnotation(String corpusId, String textId, String text, Tokenizer.Tokenization tokenization)
TextAnnotation	CuratorAnnotatorService.createAnnotatedTextAnnotation(String corpusId, String textId, String text, Tokenizer.Tokenization tokenization, Set<String> viewNames)
TextAnnotation	CuratorAnnotatorService.createBasicTextAnnotation(String corpusId, String docId, String text, Tokenizer.Tokenization tokenization)
TextAnnotation	CuratorTextAnnotationBuilder.createTextAnnotation(String corpusId, String textId, String text, Tokenizer.Tokenization tokenization)

Uses of Tokenizer.Tokenization in edu.illinois.cs.cogcomp.nlp.tokenizer

Methods in edu.illinois.cs.cogcomp.nlp.tokenizer that return Tokenizer.Tokenization

Modifier and Type	Method and Description
Tokenizer.Tokenization	StatefulTokenizer.tokenizeTextSpan(String textSpan)
Tokenizer.Tokenization	IllinoisTokenizer.tokenizeTextSpan(String text) Deprecated. given a span of text, return a list of Pair< String[], IntPair[] > corresponding to tokenized sentences, where the String[] is the ordered list of sentence tokens and the IntPair[] is the corresponding list of character offsets with respect to the original text.
Tokenizer.Tokenization	Tokenizer.tokenizeTextSpan(String textSpan) given a span of text, return a list of Pair< String[], IntPair[] > corresponding to tokenized sentences, where the String[] is the ordered list of sentence tokens and the IntPair[] is the corresponding list of character offsets with respect to the original text.

Uses of Tokenizer.Tokenization in edu.illinois.cs.cogcomp.nlp.utility

Methods in edu.illinois.cs.cogcomp.nlp.utility with parameters of type Tokenizer.Tokenization

Modifier and Type	Method and Description
TextAnnotation	TokenizerTextAnnotationBuilder.createTextAnnotation(String corpusId, String textId, String text, Tokenizer.Tokenization tokenization) A stub method that should not be called with this Builder.

Uses of Tokenizer.Tokenization in edu.illinois.cs.cogcomp.tokenizer

Methods in edu.illinois.cs.cogcomp.tokenizer that return Tokenizer.Tokenization

Modifier and Type	Method and Description
Tokenizer.Tokenization	EnglishTokenizer.tokenizeTextSpan(String textSpan) given a span of text, return a list of Pair< String[], IntPair[] > corresponding to tokenized sentences, where the String[] is the ordered list of sentence tokens and the IntPair[] is the corresponding list of character offsets with respect to the original text.
Tokenizer.Tokenization	ChineseTokenizer.tokenizeTextSpan(String text) given a span of text, return a list of Pair< String[], IntPair[] > corresponding to tokenized sentences, where the String[] is the ordered list of sentence tokens and the IntPair[] is the corresponding list of character offsets with respect to the original text.
Tokenizer.Tokenization	CharacterTokenizer.tokenizeTextSpan(String text) given a span of text, return a list of Pair< String[], IntPair[] > corresponding to tokenized sentences, where the String[] is the ordered list of sentence tokens and the IntPair[] is the corresponding list of character offsets with respect to the original text.
Tokenizer.Tokenization	WhiteSpaceTokenizer.tokenizeTextSpan(String textSpan) given a span of text, return a list of Pair< String[], IntPair[] > corresponding to tokenized sentences, where the String[] is the ordered list of sentence tokens and the IntPair[] is the corresponding list of character offsets with respect to the original text.
Tokenizer.Tokenization	StanfordAnalyzer.tokenizeTextSpan(String textSpan) given a span of text, return a list of Pair< String[], IntPair[] > corresponding to tokenized sentences, where the String[] is the ordered list of sentence tokens and the IntPair[] is the corresponding list of character offsets with respect to the original text.
Tokenizer.Tokenization	ThaiTokenizer.tokenizeTextSpan(String textSpan) given a span of text, return a list of Pair< String[], IntPair[] > corresponding to tokenized sentences, where the String[] is the ordered list of sentence tokens and the IntPair[] is the corresponding list of character offsets with respect to the original text.