I have read about the Java implementation but I cannot remember this being defined. How is the tokentype file used by the Lexer? Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/