andyr.jtokeniser
Class WhiteSpaceTokeniser

java.lang.Object
  extended by andyr.jtokeniser.Tokeniser
      extended by andyr.jtokeniser.WhiteSpaceTokeniser

public class WhiteSpaceTokeniser
extends Tokeniser

The WhiteSpaceTokeniser class a basic tokeniser that uses whitespace to separate tokens from the input string. Whitespace characters are "\t\n\r\f": the space character, the tab character, the newline character, the carriage-return character, and the form-feed character.

The following is one example of the use of the tokenizer. The code:

     WhiteSpaceTokeniser wst = new WhiteSpaceTokeniser("the cat sat on the mat");
     while (wst.hasMoreTokens()) {
         System.out.println(wst.nextToken());
     }
 

prints the following output:

     the
     sat
     on
     the
     mat
 

Version:
1.2 (01-Aug-2005)
Author:
Andrew Roberts

Field Summary
 
Fields inherited from class andyr.jtokeniser.Tokeniser
currentTokenPosition, tokens
 
Constructor Summary
WhiteSpaceTokeniser(java.lang.String input)
          Creates a WhiteSpaceTokeniser that tokenises the input.
 
Method Summary
 
Methods inherited from class andyr.jtokeniser.Tokeniser
countTokens, getTokens, hasMoreTokens, nextToken, numberOfTokens
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

WhiteSpaceTokeniser

public WhiteSpaceTokeniser(java.lang.String input)
Creates a WhiteSpaceTokeniser that tokenises the input. The tokeniser uses the default delimiter set, which is "\t\n\r\f": the space character, the tab character, the newline character, the carriage-return character, and the form-feed character. Delimiter characters themselves will not be treated as tokens.

Parameters:
input - a string from which the tokens will be extracted.