[antlr-interest] Lexing problem I cannot resolve

Gavin Lambert antlr at mirality.co.nz
Sun Aug 3 13:39:02 PDT 2008


At 08:11 4/08/2008, Carter Cheng wrote:
 >Thanks for the replies both. I have been trying this and it 
still
 >does not seem to lex properly. Am I still doing something wrong? 

 >This is with ANTLR 3.0.1
[...]
 >INT	
 >	: Digit+ ( ('..') =>
 >        | '.' Digit* { $type = FLOAT; } )?
 >
 >	| ( '0' ('x'|'X') ) => '0' ('x'|'X') (Digit|'a'..'f'|'A'..'F')+
 >        | ('.' Digit+) => '.' Digit+ { $type = FLOAT; }
 >	;

Hmm.  First of all, make sure that the FLOAT rule is a fragment 
rule -- doesn't matter what's in it (since it'll never be called), 
but it must not be a top-level candidate:
   fragment FLOAT: Digit;

If that doesn't help, then you might need to merge the DOTDOT and 
INT rules, as I said before.  But where and how is it failing?

(A useful thing to do here is to try running the lexer in 
isolation and dump the generated tokens to screen/file, so you can 
see what tokens are being generated and what each token contains.)



More information about the antlr-interest mailing list