com.go.tea.compiler
Class Token

java.lang.Object
  |
  +--com.go.tea.compiler.Token
All Implemented Interfaces:
java.io.Serializable

public class Token
extends java.lang.Object
implements java.io.Serializable

A Token represents the smallest whole element of a source file. Tokens are produced by a Scanner.

Version:
46 , 5/31/01
Author:
Brian S O'Neill
See Also:
Serialized Form

Field Summary
static int AND
          Token ID for the and keyword: 'and'
static int ASSIGN
          Token ID for the assignment operator: '='
static int BREAK
          Token ID for the break keyword: 'break'
static int CALL
          Token ID for the call keyword: 'call'
static int COMMA
          Token ID for the comma: ','
static int COMMENT
          Token ID for a single-line or multi-line comment.
static int CONCAT
          Token ID for the concatenation operator: '&'
static int DIV
          Token ID for the division operator: '/'
static int DOT
          Token ID for the dot: '.'
static int DOTDOT
          Token ID for the double dot: '..'
static int DOUBLE_HASH
          Token ID for the double hash: '##'
static int ELLIPSIS
          Token ID for the ellipsis: '...'
static int ELSE
          Token ID for the else keyword: 'else'
static int ENTER_CODE
          Token ID for the start of a code region.
static int ENTER_TEXT
          Token ID for the start of a text region.
static int EOF
          Token ID for the end of file.
static int EQ
          Token ID for the equal operator: '=='
static int FALSE
          Token ID for the false literal: 'false'
static int FOREACH
          Token ID for the for-each keyword: 'foreach'
static int GE
          Token ID for the greater than or equal operator: '>='
static int GT
          Token ID for the greater than operator: '>'
static int HASH
          Token ID for the hash: '#'
static int IDENT
          Token ID for an identifier.
static int IF
          Token ID for the if keyword: 'if'
static int IN
          Token ID for the in keyword: 'in'
static int ISA
          Token ID for the is-a keyword: 'isa'
static int LBRACE
          Token ID for the left brace: '{'
static int LBRACK
          Token ID for the left bracket: '['
static int LE
          Token ID for the less than or equal operator: '<='
static int LPAREN
          Token ID for the left parenthesis: '('
static int LT
          Token ID for the less than operator: '<'
static int MINUS
          Token ID for the minus operator: '-'
static int MOD
          Token ID for the modulus operator: '%'
static int MULT
          Token ID for the multiplication operator: '*'
static int NE
          Token ID for the not equal operator: '!='
static int NOT
          Token ID for the not keyword: 'not'
static int NULL
          Token ID for the null literal: 'null'
static int NUMBER
          Token ID for a number literal.
static int OR
          Token ID for the or keyword: 'or'
static int PLUS
          Token ID for the plus operator: '+'
static int RBRACE
          Token ID for the right brace: '}'
static int RBRACK
          Token ID for the right bracket: ']'
static int REVERSE
          Token ID for the reverse keyword: 'reverse'
static int RPAREN
          Token ID for the right parenthesis: ')'
static int SEMI
          Token ID for the semi-colon: ';'
static int SLASH
          Token ID for the forward slash: '/'
static int STRING
          Token ID for a string literal.
static int TEMPLATE
          Token ID for the template keyword: 'template'
static int TRUE
          Token ID for the true literal: 'true'
static int UNKNOWN
          Token ID for an unknown token.
 
Constructor Summary
Token(SourceInfo info, int tokenID)
           
 
Method Summary
 void dump()
          Dumps the contents of this Token to System.out.
 void dump(java.io.PrintStream out)
          Dumps the contents of this Token.
static int findReservedWordID(java.lang.StringBuffer word)
          If the given StringBuffer starts with a valid token type, its ID is returned.
 java.lang.String getCode()
          Token code is non-null, and is exactly the same as the name for its ID.
 double getDoubleValue()
          Only valid if token is a number.
 float getFloatValue()
          Only valid if token is a number.
 int getID()
          Returns the ID of this Token, which identifies what type of token it is.
 java.lang.String getImage()
          Token image represents what a static token looks like in a source file.
 int getIntValue()
          Only valid if token is a number.
 long getLongValue()
          Only valid if token is a number.
static int getMatchingBracket(int id)
          If the given id is a bracket: (,[,{,},],) then the matching bracket's id is returned.
 int getNumericType()
          Only valid if token is a number.
 SourceInfo getSourceInfo()
          Returns information regarding where in the source file this token came from.
 java.lang.String getStringValue()
           
 boolean isBracket()
          Returns true if this Token is a bracket: (,[,{,},],)
static boolean isBracket(int id)
          Returns true if id is a bracket
 boolean isCloseBracket()
          Returns true if this Token is a close bracket: ),],}
static boolean isCloseBracket(int id)
          Returns true if id is a close bracket: ), ], }
 boolean isOpenBracket()
          Returns true if this Token is an open bracket: (,[,{
static boolean isOpenBracket(int id)
          Returns true if id is an open bracket: (,[,{
 boolean isOperator()
          Returns true if this Token is an operator
static boolean isOperator(int id)
          Returns true if id is an operator
 boolean isReservedWord()
          Returns true if this Token is a reserved word.
static boolean isReservedWord(int id)
          Returns true if id is a reserved word
 java.lang.String toString()
           
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
 

Field Detail

UNKNOWN

public static final int UNKNOWN
Token ID for an unknown token.

EOF

public static final int EOF
Token ID for the end of file.

COMMENT

public static final int COMMENT
Token ID for a single-line or multi-line comment.

ENTER_CODE

public static final int ENTER_CODE
Token ID for the start of a code region.

ENTER_TEXT

public static final int ENTER_TEXT
Token ID for the start of a text region.

STRING

public static final int STRING
Token ID for a string literal.

NUMBER

public static final int NUMBER
Token ID for a number literal.

IDENT

public static final int IDENT
Token ID for an identifier.

LPAREN

public static final int LPAREN
Token ID for the left parenthesis: '('

RPAREN

public static final int RPAREN
Token ID for the right parenthesis: ')'

LBRACE

public static final int LBRACE
Token ID for the left brace: '{'

RBRACE

public static final int RBRACE
Token ID for the right brace: '}'

LBRACK

public static final int LBRACK
Token ID for the left bracket: '['

RBRACK

public static final int RBRACK
Token ID for the right bracket: ']'

SEMI

public static final int SEMI
Token ID for the semi-colon: ';'

COMMA

public static final int COMMA
Token ID for the comma: ','

DOT

public static final int DOT
Token ID for the dot: '.'

DOTDOT

public static final int DOTDOT
Token ID for the double dot: '..'

ELLIPSIS

public static final int ELLIPSIS
Token ID for the ellipsis: '...'

SLASH

public static final int SLASH
Token ID for the forward slash: '/'

HASH

public static final int HASH
Token ID for the hash: '#'

DOUBLE_HASH

public static final int DOUBLE_HASH
Token ID for the double hash: '##'

LT

public static final int LT
Token ID for the less than operator: '<'

LE

public static final int LE
Token ID for the less than or equal operator: '<='

EQ

public static final int EQ
Token ID for the equal operator: '=='

GE

public static final int GE
Token ID for the greater than or equal operator: '>='

GT

public static final int GT
Token ID for the greater than operator: '>'

NE

public static final int NE
Token ID for the not equal operator: '!='

CONCAT

public static final int CONCAT
Token ID for the concatenation operator: '&'

PLUS

public static final int PLUS
Token ID for the plus operator: '+'

MINUS

public static final int MINUS
Token ID for the minus operator: '-'

MULT

public static final int MULT
Token ID for the multiplication operator: '*'

DIV

public static final int DIV
Token ID for the division operator: '/'

MOD

public static final int MOD
Token ID for the modulus operator: '%'

ASSIGN

public static final int ASSIGN
Token ID for the assignment operator: '='

NULL

public static final int NULL
Token ID for the null literal: 'null'

TRUE

public static final int TRUE
Token ID for the true literal: 'true'

FALSE

public static final int FALSE
Token ID for the false literal: 'false'

NOT

public static final int NOT
Token ID for the not keyword: 'not'

OR

public static final int OR
Token ID for the or keyword: 'or'

AND

public static final int AND
Token ID for the and keyword: 'and'

IF

public static final int IF
Token ID for the if keyword: 'if'

ELSE

public static final int ELSE
Token ID for the else keyword: 'else'

ISA

public static final int ISA
Token ID for the is-a keyword: 'isa'

FOREACH

public static final int FOREACH
Token ID for the for-each keyword: 'foreach'

IN

public static final int IN
Token ID for the in keyword: 'in'

REVERSE

public static final int REVERSE
Token ID for the reverse keyword: 'reverse'

TEMPLATE

public static final int TEMPLATE
Token ID for the template keyword: 'template'

CALL

public static final int CALL
Token ID for the call keyword: 'call'

BREAK

public static final int BREAK
Token ID for the break keyword: 'break'
Constructor Detail

Token

public Token(SourceInfo info,
             int tokenID)
Method Detail

isReservedWord

public static final boolean isReservedWord(int id)
Returns true if id is a reserved word
Parameters:
id - The Token id to test

isOperator

public static final boolean isOperator(int id)
Returns true if id is an operator
Parameters:
id - The Token id to test

isBracket

public static final boolean isBracket(int id)
Returns true if id is a bracket
Parameters:
id - The Token id to test

isOpenBracket

public static final boolean isOpenBracket(int id)
Returns true if id is an open bracket: (,[,{
Parameters:
id - The Token id to test

isCloseBracket

public static final boolean isCloseBracket(int id)
Returns true if id is a close bracket: ), ], }
Parameters:
id - The Token id to test

getMatchingBracket

public static final int getMatchingBracket(int id)
If the given id is a bracket: (,[,{,},],) then the matching bracket's id is returned. If id is not a bracket, then -1 is returned.

findReservedWordID

public static int findReservedWordID(java.lang.StringBuffer word)
If the given StringBuffer starts with a valid token type, its ID is returned. Otherwise, the token ID UNKNOWN is returned.

dump

public final void dump()
Dumps the contents of this Token to System.out.

dump

public final void dump(java.io.PrintStream out)
Dumps the contents of this Token.
Parameters:
out - The PrintStream to write to.

getID

public final int getID()
Returns the ID of this Token, which identifies what type of token it is.

isReservedWord

public final boolean isReservedWord()
Returns true if this Token is a reserved word.

isBracket

public final boolean isBracket()
Returns true if this Token is a bracket: (,[,{,},],)

isOpenBracket

public final boolean isOpenBracket()
Returns true if this Token is an open bracket: (,[,{

isCloseBracket

public final boolean isCloseBracket()
Returns true if this Token is a close bracket: ),],}

isOperator

public final boolean isOperator()
Returns true if this Token is an operator

getImage

public java.lang.String getImage()
Token image represents what a static token looks like in a source file. Token image is null if token is a string, number or identifier because these tokens don't have static images.

getCode

public java.lang.String getCode()
Token code is non-null, and is exactly the same as the name for its ID.

getSourceInfo

public final SourceInfo getSourceInfo()
Returns information regarding where in the source file this token came from.

getStringValue

public java.lang.String getStringValue()

getNumericType

public int getNumericType()
Only valid if token is a number. Returns 0 if token is not a number or is an invalid number. Returns 1 for int, 2 for long, 3 for float and 4 for double. The token ID for all numbers (even invalid ones) is NUMBER.
Returns:
0, 1, 2, 3 or 4.

getIntValue

public int getIntValue()
Only valid if token is a number.

getLongValue

public long getLongValue()
Only valid if token is a number.

getFloatValue

public float getFloatValue()
Only valid if token is a number.

getDoubleValue

public double getDoubleValue()
Only valid if token is a number.

toString

public java.lang.String toString()
Overrides:
toString in class java.lang.Object