[antlr-interest] Re: lexer labels in v3

[Oliver Zeigermann]

Unfortunately, this does not work when the '!' operator is used to
suppres certain parts of the token. Right now I have no idea how to
fix this :(

Any ideas?

Oliver

2005/9/25, Terence Parr <parrt at cs.usfca.edu>:
> Oliver, nice work.  Labels on lexer fragment rule refs now work great:
>
> lexer grammar T;
>
> A : 'a' x=B y="void" z='x' ;
>
> fragment
> B : 'b' ;
>
> It still works, but is slightly inefficient when B is not a fragment.
>
> I added similar stuff to char and string refs.  Note that labelling a
> string should be a token label
>
> x="void"
>
> should define x as a Token right whereas
>
> c='x'
>
> has label c as an int?
>
> Another difference is that labels will be unique to the alternative
> as they can have different types.  In parser/tree parser rules,
> everything is one type: Token or AST so I collect and put labels at
> the start of the rule so you can ref them like this:
>
> ( x:A | x:B )
> {print x;}
>
> pretty handy.  Are you ok with the inconsistency?  It's rare to label
> stuff in the lexer so this is probably ok.
>
> The code block for A is (comment auto generated):
>
>          // t.g:3:5: 'a' x= B y= "void" z= 'x'
>          {
>          match('a');
>          int xStart = getCharIndex();
>          mB();
>          Token x = new CommonToken(input, Token.INVALID_TOKEN_TYPE,
> Token.DEFAULT_CHANNEL, xStart, getCharIndex()-1);
>
>          int yStart = getCharIndex();
>          match("void");
>          Token y = new CommonToken(input, Token.INVALID_TOKEN_TYPE,
> Token.DEFAULT_CHANNEL, yStart, getCharIndex()-1);
>
>          int z = input.LA(1);
>          match('x');
>          }
>
> Oh, wildcard labels work too :)
>
> Ter
>
> On Sep 12, 2005, at 6:39 AM, Oliver Zeigermann wrote:
>
> > Modifying template lexerRuleRef in Java.stg
> > (org/antlr/codegen/templates/) to this
> >
> > lexerRuleRef(label,rule,args) ::= <<
> > <if(label)>
> > int <label>Start = getCharIndex();<\n>
> > m<rule>(<args>);<\n>
> > Token <label> = new CommonToken(input, Token.INVALID_TOKEN_TYPE,
> > Token.DEFAULT_CHANNEL, <label>Start, getCharIndex()-1);<\n>
> > <else>
> > m<rule>(<args>);<\n>
> > <endif>
> >
> >>>
> >>>
> >
> > allowed me to have labled lexer fragments like
> >
> > name=GENERIC_ID
> >
> > ...
> >
> > fragment GENERIC_ID     : ... ;
> >
> > Oliver
> >
> >
> > 2005/9/12, Oliver Zeigermann <oliver.zeigermann at gmail.com>:
> >
> >> In codegen.g, rule atom the attribute "label" is set for
> >> lexerStringRef, charRef and lexerRuleRef while there is no attribute
> >> label for these rules. This causes an error upon generation.
> >>
> >> Obvious solution: either remove code that sets "label" (already done
> >> at certain parts of rule atom) or add attribute "label" to the
> >> templates.
> >>
> >> Oliver
> >>
> >
>
> --
> CS Professor & Grad Director, University of San Francisco
> Creator, ANTLR Parser Generator, http://www.antlr.org
> Cofounder, http://www.jguru.com
>
>