[antlr-interest] Fragments and setText appear to not work at all - not even with code from the book

Austin Hastings Austin_Hastings at Yahoo.com
Wed Nov 21 02:44:37 PST 2007


I think the underlying source of your angst is the fact that setText() 
works against a non-scoped (effectively: global) object. If you are 
recursing in your token rule, all the calls to setText() update the same 
value. Unless your rule is structured to make use of this somehow (such 
as by calling getText(), I guess) then it probably won't DWYW.

I recently got burned by a similar problem, calling a full-fledged token 
method (that used setText()) when I should have made a fragment.

I think the right idiom to use is this:

TOKEN: TokenFragment { setText(...); } ;
fragment TokenFragment: ... recognize without updating ... ;

In your case, you may want to investigate creating a scope, or a return 
value for the fragment, or using a stack-like construct of your own 
devising.

Anyway, you're right -- the example doesn't work. Welcome to hell, 
here's your accordion.

=Austin


Harald Mueller wrote:
>> Anyway, what you refer to is not a 
>> bug in the sense that it was a deliberate design decision for the sake 
>> of performance. [...] In other words, "settext" that 
>> you do in a fragment rule is irrelevant.
>>     
>
> And, anyway, the following also does not work (it replaces the main method and the complete grammar in my original posting):
>
> ------------------
>   public static void main(String[] args) throws Exception {
>     run("{ a { b c }}");
>   }
> }
>
> a : m=CODE { System.out.println("m:" + m.getText()); }
>   ;
>
> CODE
>     : '{' ( x=CODE | ~('{'|'}') )* '}'
>         {
>             setText(getText().substring(1, getText().length()));
>             //C#: $text = $text.Substring(1, getText().length()-1);
>         }
>     ;
>
> ------------------
>
> No fragments here, only a simple recursive rule. The output is, as before:
>
> { a { b c }} ==> a { b c }
>
> So setText doesn't seem to work also for a non-fragment lexer rule called inside another lexer rule ... It appears to me that all this should be checked and output by the ANTLR tool (which probably knows about all these rules), shouldn't it? (ok - I know: it doesn't look into the code blocks, so it allows you to do anything there ... it's at your own risk ... but that means at least documentation, documentation, documentation ...).
>
> Regards
> Harald
>
>   



More information about the antlr-interest mailing list