[antlr-interest] Accessing the token list from a lexer?

[John B. Brodie]

>In ANTLR 2.7.x is there a clean way to get the token list out of a
>Lexer generated by ANTLR? I'm working in Java if that makes a
>difference.
>
>I can see hackish ways (access the literals hashtable, and extract the
>string byte-by-byte from the ANTLRHashString objects), but nothing
>cleaner is jumping out at me.

Do you want a collection that contains the set of tokens a particular lexer is
able to recognize?

   -- if so, maybe the *TokenTypes.txt files generated by Antlr can help.

Or do you want a collection that contains the set of tokens that are
produced by a lexer when lexing a particular input file?

   -- if so, just construct a lexer on the input file and then repeatedly call
      its next() method to obtain each token recognized (just like the parser
      does).

Hope this helps...
   -jbb