[antlr-interest] bug in 3.0b6: identifier/keyword or underscoreproblem?

[Jim Idle]

Why do you wish to have filter lexer though? Why not just have a catch
all rule at the end that matches anything and just assigns the channel
to hidden? It seems you might be using filters for something that
doesn't want to be done with a filter ;-)


THROWAWAY : . {$channel=hidden;}

Then use a normal parser/lexer combination?

Jim

-----Original Message-----
From: antlr-interest-bounces at antlr.org
[mailto:antlr-interest-bounces at antlr.org] On Behalf Of Martin d'Anjou
Sent: Monday, February 26, 2007 4:13 AM
To: antlr-interest at antlr.org
Subject: [antlr-interest] bug in 3.0b6: identifier/keyword or
underscoreproblem?

Hi,

I've been strugling with this for several days. It looks like 3.0b6
can't 
tell an identifier from a keyword, or has a problem with underscores.
Here 
is how I produce the problem:

1) input text

int id;
int int_id;
int _int_id;

2) Lexer

lexer grammar DUMMY_Lexer;
options { filter=true; }

INT          : 'int' ;
SEMI         : ';' ;
WS           :  (  ' '| '\t'| '\r' | '\n' )+ {$channel=HIDDEN;} ;
IDENTIFIER   : ('a'..'z'|'A'..'Z'|'_')+;

3) Parser

parser grammar DUMMY_Parser;
options {
   tokenVocab=DUMMY_Lexer;
}

source_text :
   { System.out.println("Weird lexer"); }
   int_defs+
   ;

int_defs :
   INT            { System.out.print("int "); }
   id=IDENTIFIER  { System.out.print($id.text); }
   SEMI           { System.out.println(";"); }
   ;

4) The output and the error I get

Weird lexer
int id;
int line 2:4 mismatched input 'int' expecting IDENTIFIER
int;
int _int_id;


The lexer is not greedy or something is wrong with the underscore, I
don't 
know.

Help!!!
Martin