WordsToTokens (illinois-cogcomp-nlp 3.1.29 API)

java.lang.Object
- edu.illinois.cs.cogcomp.lbjava.nlp.seg.WordsToTokens

All Implemented Interfaces:

edu.illinois.cs.cogcomp.lbjava.parse.Parser
```
public class WordsToTokens
extends Object
implements edu.illinois.cs.cogcomp.lbjava.parse.Parser
```
This parser calls another parser that returns LinkedVectors of Words, converts the Words to Tokens, and returns LinkedVectors of Tokens.

Author:

Nick Rizzolo

Field Summary

Fields
Modifier and Type Field and Description

protected edu.illinois.cs.cogcomp.lbjava.parse.Parser parser
A parser that returns LinkedVectors of Words.

Fields
Modifier and Type	Field and Description
`protected edu.illinois.cs.cogcomp.lbjava.parse.Parser`	`parser` A parser that returns `LinkedVector`s of `Word`s.

Constructor Summary

Constructors
Constructor and Description

WordsToTokens(edu.illinois.cs.cogcomp.lbjava.parse.Parser p)
Creates the parser.

Constructors
Constructor and Description
`WordsToTokens(edu.illinois.cs.cogcomp.lbjava.parse.Parser p)` Creates the parser.

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`void`	`close()` Frees any resources this parser may be holding.
`static edu.illinois.cs.cogcomp.lbjava.parse.LinkedVector`	`convert(edu.illinois.cs.cogcomp.lbjava.parse.LinkedVector v)` Given a `LinkedVector` containing `Word`s, this method creates a new `LinkedVector` containing `Token`s.
`Object`	`next()` Returns the next `LinkedVector` of `Token`s.
`void`	`reset()` Sets this parser back to the beginning of the raw data.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - parser
```
protected edu.illinois.cs.cogcomp.lbjava.parse.Parser parser
```
    A parser that returns LinkedVectors of Words.
- Constructor Detail
  - WordsToTokens
```
public WordsToTokens(edu.illinois.cs.cogcomp.lbjava.parse.Parser p)
```
    Creates the parser.
    
    Parameters:
    
    p - A parser that returns LinkedVectors of Words.
- Method Detail
  - next
```
public Object next()
```
    Returns the next LinkedVector of Tokens.
    
    Specified by:
    
    next in interface edu.illinois.cs.cogcomp.lbjava.parse.Parser
    
    Returns:
    
    The next LinkedVector of Tokens parsed, or null if there are no more children in the stream.
  - convert
```
public static edu.illinois.cs.cogcomp.lbjava.parse.LinkedVector convert(edu.illinois.cs.cogcomp.lbjava.parse.LinkedVector v)
```
    Given a LinkedVector containing Words, this method creates a new LinkedVector containing Tokens.
    
    Parameters:
    
    v - A LinkedVector of Words.
    
    Returns:
    
    A LinkedVector of Tokens corresponding to the input Words.
  - reset
```
public void reset()
```
    Sets this parser back to the beginning of the raw data.
    
    Specified by:
    
    reset in interface edu.illinois.cs.cogcomp.lbjava.parse.Parser
  - close
```
public void close()
```
    Frees any resources this parser may be holding.
    
    Specified by:
    
    close in interface edu.illinois.cs.cogcomp.lbjava.parse.Parser

Class WordsToTokens

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

parser

Constructor Detail

WordsToTokens

Method Detail

next

convert

reset

close