[antlr-interest] Combined grammars behave differently than separate parser/lexer grammars

[Ron Hunter-Duvar]

As a work around for this, I wrote a short script that sorts each of the 
*.token files and diffs them. The only tokens that should show up as 
added in the parser are those that are defined in the tokens section at 
the top (I only use that for token types being used as imaginary AST 
nodes, and I distinguish these by naming convention). If all your token 
types are supposed to be defined in the lexer, then there should be no 
diffs.

Ron


Stevenson, Todd (GE Healthcare) wrote:
> I recently split a combined grammar that I had built into separate
> parser and lexer grammars.  I noticed  that in the combined grammar when
> I attempt to reference an undefined token, Antlr throws an error.
> However, after I split this grammar into separate lexer and parser
> grammars, Antlr allowed me to reference lexer tokens in the parser
> grammar not defined by the lexer grammar.  These become very difficult
> to debug.   Am I missing something, or is the the correct behavior?
>  
> thanks.
>  
>  
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>
>   

-- 
Ron Hunter-Duvar | Software Developer V | 403-272-6580
Oracle Service Engineering
Gulf Canada Square 401 - 9th Avenue S.W., Calgary, AB, Canada T2P 3C5

All opinions expressed here are mine, and do not necessarily represent
those of my employer.