net.sf.eos.analyzer
Class WhitespaceTokenizer

java.lang.Object
  net.sf.eos.analyzer.TokenFilter
      net.sf.eos.analyzer.WhitespaceTokenizer

All Implemented Interfaces:

ResettableTokenizer, Tokenizer

public final class WhitespaceTokenizer
extends TokenFilter
implements ResettableTokenizer

Tokenized a sequence of chars at whitespaces. Wrapper around Lucenes WhitespaceTokenizer.

Author:

Sascha Kohlmann

Constructor Summary

WhitespaceTokenizer()
Creates a new instance.

WhitespaceTokenizer(CharSequence text)
Creates a new instance for a char sequence.

WhitespaceTokenizer(Tokenizer source)
Creates a new instance.

Method Summary

protected Tokenizer	getSource() Return value may be null.
Token	next() Implementations should use TokenFilter.getSource() to fetch the source Tokenizer an handle the resulting Token.
void	reset(CharSequence input) Inits the tokenizer with new input data.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

WhitespaceTokenizer

public WhitespaceTokenizer(Tokenizer source)

Creates a new instance.

Parameters:

source - a source filter

WhitespaceTokenizer

public WhitespaceTokenizer()

Creates a new instance.

WhitespaceTokenizer

public WhitespaceTokenizer(CharSequence text)

Creates a new instance for a char sequence.

Parameters:

text - the sequence to tokenize

Method Detail

next

public Token next()
           throws TokenizerException

Description copied from class: TokenFilter

Implementations should use TokenFilter.getSource() to fetch the source Tokenizer an handle the resulting Token.

Specified by:

next in interface Tokenizer

Specified by:

next in class TokenFilter

Returns:

the next token or null

Throws:

TokenizerException - if an error occurs

reset

public void reset(CharSequence input)
           throws TokenizerException

Description copied from interface: ResettableTokenizer

Inits the tokenizer with new input data.

Specified by:

reset in interface ResettableTokenizer

Parameters:

input - represents new input data for the tokenizer.

Throws:

TokenizerException

getSource

protected Tokenizer getSource()

Return value may be null.

Overrides:

getSource in class TokenFilter

Returns:

may be null