net.sf.eos.analyzer
Interface Token

All Superinterfaces:

Metadata

All Known Implementing Classes:

AbstractToken

public interface Token
extends Metadata

A Token represents a part of a tokenized text.

Author:

Sascha Kohlmann

See Also:

Tokenizer

Field Summary

static String

DEFAULT_TYPE The default type of a token.

Method Summary

CharSequence	getTokenText() Returns the text of a token.
String	getType() Returns the type of the token.

Methods inherited from interface net.sf.eos.Metadata

getMeta

Field Detail

DEFAULT_TYPE

static final String DEFAULT_TYPE

The default type of a token.

See Also:

Constant Field Values

Method Detail

getTokenText

CharSequence getTokenText()

Returns the text of a token. The text may be a single word or a sequence of words, e.g. a sentence.

Returns:

the character sequence of the token

getType

String getType()

Returns the type of the token.

Returns:

the type of a token.