Uses of Interface
net.sf.eos.analyzer.Tokenizer

Packages that use Tokenizer

net.sf.eos.analyzer	The package contains classes and patterns to support analyzing of CharSequence.
net.sf.eos.analyzer.lucene	Contains adapter and support for reuse of Lucene Tokenizer.
net.sf.eos.entity	Support for named entity recognizer.

Uses of Tokenizer in net.sf.eos.analyzer

Subinterfaces of Tokenizer in net.sf.eos.analyzer

interface

ResettableTokenizer Implementation that are prepared for reuse should implement this interface.

Classes in net.sf.eos.analyzer that implement Tokenizer

class	CaseTokenFilter Transforms the input token to a upper or lower cased format for a given Locale.
class	ResettableTokenFilter A token filter that supports handling with resettable tokenizer.
class	SentenceTokenizer Tokenized a text into sentences.
class	StopTokenFilter Filter for stop words out of the token stream.
class	SurroundingTokenFilter The filter removes surrounding braces and other characters around a token text.
class	TokenFilter Main class to support Tokenizer chaining, also known as decorator pattern.
class	WhitespaceTokenizer Tokenized a sequence of chars at whitespaces.

Methods in net.sf.eos.analyzer that return Tokenizer

protected Tokenizer	WhitespaceTokenizer.getSource() Return value may be null.
protected Tokenizer	TokenFilter.getSource() Returns the source Tokenizer.

Constructors in net.sf.eos.analyzer with parameters of type Tokenizer

CaseTokenFilter(Tokenizer source)
Creates a new instance for lowercase with the default Locale.

CaseTokenFilter(Tokenizer source, boolean upper)
Creates a new instance with the default Locale.

CaseTokenFilter(Tokenizer source, Locale locale)
Creates a new instance for lowercase with the given Locale.

CaseTokenFilter(Tokenizer source, Locale locale, boolean upper)
Creates a new instance.

ResettableTokenFilter(Tokenizer source, ResettableTokenizer resettable)
Creates a new instance with the source using described in TokenFilter.

StopTokenFilter(Tokenizer source, Set<CharSequence> stopWords)
Creates a new instance.

SurroundingTokenFilter(Tokenizer tokenizer)

TokenFilter(Tokenizer source)
Creates a new instance.

WhitespaceTokenizer(Tokenizer source)
Creates a new instance.

Uses of Tokenizer in net.sf.eos.analyzer.lucene

Classes in net.sf.eos.analyzer.lucene that implement Tokenizer

class

LuceneTokenizerWrapper Wraps the functionality of the εοs Tokenizer for the reuse of Lucene tokenizer.

Constructors in net.sf.eos.analyzer.lucene with parameters of type Tokenizer

LuceneTokenizerWrapper(Tokenizer source, org.apache.lucene.analysis.Tokenizer tokenizer)
Creates a new wrapper.

Uses of Tokenizer in net.sf.eos.entity

Subinterfaces of Tokenizer in net.sf.eos.entity

interface	DictionaryBasedEntityRecognizer The DictionaryBasedEntityRecognizer uses a Map to recognize entities in a text.
interface	EntityRecognizer The strategy for entity recognition is not defined.

Classes in net.sf.eos.entity that implement Tokenizer

class	AbstractDictionaryBasedEntityRecognizer An implementation of a @code EntityRecognizer} identifies entities in a text.
class	SimpleLongestMatchDictionaryBasedEntityRecognizer A simple matcher for named entities.

Methods in net.sf.eos.entity with parameters of type Tokenizer

static DictionaryBasedEntityRecognizer	AbstractDictionaryBasedEntityRecognizer.newInstance(Tokenizer source) Creates a new instance of a of the recognizer.
static DictionaryBasedEntityRecognizer	AbstractDictionaryBasedEntityRecognizer.newInstance(Tokenizer source, Configuration config) Creates a new instance of a of the recognizer.

Constructors in net.sf.eos.entity with parameters of type Tokenizer

AbstractDictionaryBasedEntityRecognizer(Tokenizer source)

SimpleLongestMatchDictionaryBasedEntityRecognizer(Tokenizer source)
Creates a new instance.