[antlr-interest] Another parsing question

[Johannes Luber]

Carter Cheng schrieb:
> The difficulty is with the language I am working with in the first case it should be two tokens ']' ')' but the second case it should be one token '])' without intervening whitespace between the ']' and ')'. 
> 
> The only way I can see of solving this problem is to make white space explicit in the grammar. I.e. litter my rules with whitespace tokens and omit a whitespace token in the case when i expect a '])'. Is this the correct way to do this with ANTLRv3?
> 
> Regards,
> 
> Carter.

You can check manually, if the indizes of the two tokens differ by only 
one. In that case no whitespace/comment/etc. is between them. 
$RBRACKET.index == $RPAREN.index -1 should work (replace the token names 
with the ones, you used).

Johannes
> 
> 
> --- On Mon, 8/4/08, Gavin Lambert <antlr at mirality.co.nz> wrote:
> 
>> From: Gavin Lambert <antlr at mirality.co.nz>
>> Subject: Re: [antlr-interest] Another parsing question
>> To: carter_cheng at yahoo.com, antlr-interest at antlr.org
>> Date: Monday, August 4, 2008, 2:45 PM
>> At 08:45 5/08/2008, Carter Cheng wrote:
>>  >Thanks everyone for the help thus far. This is likely
>> another
>>  >beginners question concerning the behavior of the
>> lexer/parser
>>  >interactions in ANTLRv3. The question I have is are
>> the tokens
>>  >requested by the parser context dependent? i.e. is a
>> different
>>  >(N)DFA constructed depending on which tokens the
>> parser expects 
>> to
>>  >see in the stream or is there only 1 N(DFA) with a
>> series of 
>> stop
>>  >states.
>>
>> Lexing is all done up front with no input from the parser
>> at all.
>>
>>  >(a[1]); // lookup the index 1 in the variable array a
>>  >
>>  >([ 1 : 2 ]); // construct a mapping value with key :
>> value in 
>> this
>>  >case 1 : 2
>>  >
>>  >The question I have is how can ANTLRv3 know whether ])
>> is a 
>> single
>>  >token or two tokens?
>>
>> Depends on how you've defined the lexer rules :)  But
>> normally 
>> you'd want to define it as two tokens.
> 
> 
>       
>