[antlr-interest] Problems with string literals as tokens in parser rules

[Bryan Ewbank]

The way string literals work is that you must have a lexer rule that matches
them - then the string table is used to replace the node type.  In your
example, the only token recognized by the lexer starts with 'd', and
therefore the string "abc" will not be recognized.

To see this in action, change the lexer thus:

> class MyLexer extends Lexer;
> DUMMY : ('a'..'z')* ;

You will now see that DUMMY matches any sequence of letters, but if you look
at the generated code you will see that the string table is used to change
the type of the token node returned to the parser.

The classic use of stringLiterals is for keywords in a language - write a
rule to match all identifiers, then rely on the string table to split out
those tokens which are keywords ("while", "for", ...).

Hope this helps,
- Bryan Ewbank



 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
    antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/