[antlr-interest] Another parsing question

[Carter Cheng]

The difficulty is with the language I am working with in the first case it should be two tokens ']' ')' but the second case it should be one token '])' without intervening whitespace between the ']' and ')'. 

The only way I can see of solving this problem is to make white space explicit in the grammar. I.e. litter my rules with whitespace tokens and omit a whitespace token in the case when i expect a '])'. Is this the correct way to do this with ANTLRv3?

Regards,

Carter.


--- On Mon, 8/4/08, Gavin Lambert <antlr at mirality.co.nz> wrote:

> From: Gavin Lambert <antlr at mirality.co.nz>
> Subject: Re: [antlr-interest] Another parsing question
> To: carter_cheng at yahoo.com, antlr-interest at antlr.org
> Date: Monday, August 4, 2008, 2:45 PM
> At 08:45 5/08/2008, Carter Cheng wrote:
>  >Thanks everyone for the help thus far. This is likely
> another
>  >beginners question concerning the behavior of the
> lexer/parser
>  >interactions in ANTLRv3. The question I have is are
> the tokens
>  >requested by the parser context dependent? i.e. is a
> different
>  >(N)DFA constructed depending on which tokens the
> parser expects 
> to
>  >see in the stream or is there only 1 N(DFA) with a
> series of 
> stop
>  >states.
> 
> Lexing is all done up front with no input from the parser
> at all.
> 
>  >(a[1]); // lookup the index 1 in the variable array a
>  >
>  >([ 1 : 2 ]); // construct a mapping value with key :
> value in 
> this
>  >case 1 : 2
>  >
>  >The question I have is how can ANTLRv3 know whether ])
> is a 
> single
>  >token or two tokens?
> 
> Depends on how you've defined the lexer rules :)  But
> normally 
> you'd want to define it as two tokens.