[antlr-interest] howto ignore unknown tokenstreams/recordsets
Oliver Kowalke
oliver.kowalke at gmx.de
Mon Jan 17 13:01:41 PST 2005
> This one matches everything that consists of only letters. The
> testLiterals option makes sure the items added in the tokens section
> get recognized as such (before returning from the ID rule antlr checks
> against entries in the tokens table). E.g. they get passed to the
> parser as A .. D and the unknown tokens get passed as ID. You could
> use that to make the catch all rule. At least that should be the
> general idea I think.
Hi,
thx it seams to work but I get some nondeterminism warnings. How can I
remove them? Are the warnings relevant?
thx
Oliver
-- input --
A 1 2;
B 1;
X 1 2;
C "abc";
d 1 "abc";
Y "test" 123;
-- grammar ---
...
class MyLexer extends Lexer;
options
{
k=2;
charVocabulary='\u0000'..'\u007F';
caseSensitive=false;
caseSensitiveLiterals=false;
testLiterals = false;
}
tokens
{
A="a";
B="b";
C="c";
D="d";
}
NUM : ( '0'..'9' )+;
IDENT options { testLiterals = true; } : ( 'a'..'z' ) ( '0'..'9' | 'a'..'z'
)*;
STRING : '"'!
( ~( '\'' | '"' | '\n' | '\r' ) )*
( '"'!
| // nothing -- write error message
)
;
EODS : ';';
DELIM
: ( ' '
| '\t'
| '\f'
| ( "\r\n"
| '\r'
| '\n'
)
{ newline(); }
)
{ $setType(antlr::Token::SKIP); }
;
class MyParser extends Parser;
parse : datasets EOF;
datasets : ( headers EODS )* ( section1 EODS )*;
headers : a | b | ign;
section1 : c | d | ign;
a : A NUM NUM;
b : B NUM;
c : C STRING;
d : D NUM STRING;
ign : IDENT ( ~( EODS ) )* ;
--
Sparen beginnt mit GMX DSL: http://www.gmx.net/de/go/dsl
More information about the antlr-interest
mailing list