Uses of Class
andyr.jtokeniser.Tokeniser

Packages that use Tokeniser
andyr.jtokeniser   
 

Uses of Tokeniser in andyr.jtokeniser
 

Subclasses of Tokeniser in andyr.jtokeniser
 class BreakIteratorTokeniser
          The BreakIteratorTokeniser class uses a BreakIterator to find each word instance according to a specified locale.
 class RegexSeparatorTokeniser
          The RegexSeparatorTokeniser class uses regular expressions to define the separation between tokens.
 class RegexTokeniser
          The RegexTokeniser class uses regular expressions to define a word, and tokenises according to that expression.
 class SentenceTokeniser
          The SentenceTokeniser class uses a BreakIterator to find each word instance according to a specified locale.
 class StringTokeniser
          The StringTokeniser class uses a standard StringTokenizer to find each word in the input.
 class WhiteSpaceTokeniser
          The WhiteSpaceTokeniser class a basic tokeniser that uses whitespace to separate tokens from the input string.