[antlr-interest] Another parsing question

[Carter Cheng]

Thanks this seems to work.


--- On Mon, 8/4/08, Johannes Luber <jaluber at gmx.de> wrote:

> From: Johannes Luber <jaluber at gmx.de>
> Subject: Re: [antlr-interest] Another parsing question
> To: carter_cheng at yahoo.com
> Cc: antlr-interest at antlr.org
> Date: Monday, August 4, 2008, 3:12 PM
> Carter Cheng schrieb:
> > The difficulty is with the language I am working with
> in the first case it should be two tokens ']'
> ')' but the second case it should be one token
> '])' without intervening whitespace between the
> ']' and ')'. 
> > 
> > The only way I can see of solving this problem is to
> make white space explicit in the grammar. I.e. litter my
> rules with whitespace tokens and omit a whitespace token in
> the case when i expect a '])'. Is this the correct
> way to do this with ANTLRv3?
> > 
> > Regards,
> > 
> > Carter.
> 
> You can check manually, if the indizes of the two tokens
> differ by only 
> one. In that case no whitespace/comment/etc. is between
> them. 
> $RBRACKET.index == $RPAREN.index -1 should work (replace
> the token names 
> with the ones, you used).
> 
> Johannes
> > 
> > 
> > --- On Mon, 8/4/08, Gavin Lambert
> <antlr at mirality.co.nz> wrote:
> > 
> >> From: Gavin Lambert <antlr at mirality.co.nz>
> >> Subject: Re: [antlr-interest] Another parsing
> question
> >> To: carter_cheng at yahoo.com,
> antlr-interest at antlr.org
> >> Date: Monday, August 4, 2008, 2:45 PM
> >> At 08:45 5/08/2008, Carter Cheng wrote:
> >>  >Thanks everyone for the help thus far. This
> is likely
> >> another
> >>  >beginners question concerning the behavior of
> the
> >> lexer/parser
> >>  >interactions in ANTLRv3. The question I have
> is are
> >> the tokens
> >>  >requested by the parser context dependent?
> i.e. is a
> >> different
> >>  >(N)DFA constructed depending on which tokens
> the
> >> parser expects 
> >> to
> >>  >see in the stream or is there only 1 N(DFA)
> with a
> >> series of 
> >> stop
> >>  >states.
> >>
> >> Lexing is all done up front with no input from the
> parser
> >> at all.
> >>
> >>  >(a[1]); // lookup the index 1 in the variable
> array a
> >>  >
> >>  >([ 1 : 2 ]); // construct a mapping value
> with key :
> >> value in 
> >> this
> >>  >case 1 : 2
> >>  >
> >>  >The question I have is how can ANTLRv3 know
> whether ])
> >> is a 
> >> single
> >>  >token or two tokens?
> >>
> >> Depends on how you've defined the lexer rules
> :)  But
> >> normally 
> >> you'd want to define it as two tokens.
> > 
> > 
> >       
> >