[antlr-interest] Solving Lexical Non-Determinism

[Timothy Washington]

Hey all, this is more of a strategic question I guess.
I want to be able to recognise an entire block of XML
via my lexer. However, I have some lexical
non-determinism between the XML block rule (
TOKEN_LITERAL ) and it's sub-components. And it makes
sense that upon encountering an xml declaration, for
example, does the lexer start to match on the 'PI' or
the 'TOKEN_LITERAL' rule. Is there a way to make
TOKEN_LITERAL a block of all it's sub-components - the
() brackets are not sufficient in this case: 

warnings:
warning:lexical nondeterminism between rules PI and
TOKEN_LITERAL
warning:lexical nondeterminism between rules EMPTYTAG
and TOKEN_LITERAL
warning:lexical nondeterminism between rules STARTTAG
and TOKEN_LITERAL

rule:
TOKEN_LITERAL:
    (
        (PI)?   
        (       
            ( STARTTAG 
                ( WS |  PI   | COMMENT  | CDATABLOCK  
)* 
                (TOKEN_LITERAL)*
            ENDTAG )
            |       
            (EMPTYTAG)
        )       
    ) 
    { System.out.println("TOKEN LITERAL >>> Lexer");
}; 


Tim


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com