[antlr-interest] Re: expecting IDENT, found 'x'
lgcraymer
lgc at mail1.jpl.nasa.gov
Mon Mar 8 13:33:12 PST 2004
It's either/or. You can identify them, but not as literals if you are might use them as identifiers elsewhere--use semantic predicates to
match text. Or, if you do use them as literals, then you need a parser rule that spells out these items as alternatives:
id : ID | "begin" | "end" ;
and use id everywhere you might have otherwise used ID. This is much cleaner with semantic predicate hoisting--you can have
keyword rules..
--Loring
--- In antlr-interest at yahoogroups.com, Mark Buckle <mark.buckle at i...> wrote:
>
> Hi All,
>
> I've read the docs, experimented with the tokens{} section in the lexer and
> "literals" in the parser, and still
> not quite got to grips with what is going on with tokens and identifers in
> ANTLR.
>
> What I'd like to be able to do with my parser is to treat keywords in the
> language only as keywords in the places specified.
>
> Eg
> Statement_block
> : "begin"
> ( statement SEMI ) +
> "end"
> ;
>
> and have BEGIN and END only treated as distinguished keywords when defining
> a block like this, and have them treated as identifiers elsewhere.
> From what I've read, it seems to be suggested that ANTLR can do this, but
> I've not been able to get it to work. Have I misunderstood things ?
> I'm trying to write a parser for a language which has a lot of keywords
> which aren't reserved words. I've gone through the tedium of
> doing this in the past in lex/yacc for another language, and had to add the
> ambiguous keywords to a list of identifiers, which wasn't always possible
> without triggering a lot of reduce/reduce conflicts. I was hoping that
> ANTLR, being ll(k) would be able to use its current context to inform the
> lexer
> whether something would be an identifier or keyword from the current rule
> being tried.
>
> Cheers Mark.
>
>
>
>
> -----Original Message-----
> From: mzukowski at y... [mailto:mzukowski at y...]
> Sent: 08 March 2004 17:12
> To: antlr-interest at yahoogroups.com
> Subject: RE: [antlr-interest] expecting IDENT, found 'x'
>
>
> You are using ":" (in the parser) as a literal which it really shouldn't be,
> unless it can be part of an identifier.
>
> Instead you should have this in your lexer:
> COLON : ':' ;
>
> And in the parser you should use COLON, not ":".
>
> Read the docs on literals to get an idea of what was happening and when to
> use "something" in the parser.
>
> Monty
>
> -----Original Message-----
> From: jc_lelann [mailto:jc.lelann at w...]
> Sent: Saturday, March 06, 2004 6:58 AM
> To: antlr-interest at yahoogroups.com
> Subject: [antlr-interest] expecting IDENT, found 'x'
>
> Hi !
>
> Could someone explain me why my parser returns the following message ?
>
> java Test < test.txt
> line 1:10: expecting IDENT, found 'x'
> exception: line 1:12: unexpected char: ':'
>
> I am new to antlr.
> Thanks
>
> JC
>
>
>
> My grammar :
>
> // content of my test file :
>
> // constant x : integer := 4;
> // constant x,y,z : integer := 5;
> // constant x,y,z : integer := 5+5*2;
>
>
> class TestParser extends Parser;
> options {
> buildAST = true;
> defaultErrorHandler = true; // Generate parser error handlers
> }
>
> mytext :
> (constant_declaration)* EOF! ;
>
> constant_declaration :
> CONSTANT! identifier_list ":"! subtype_indication ":="! expression
> ";"! ; identifier_list :
> identifier (","! identifier)*
> ;
> identifier :
> IDENT
> ;
> subtype_indication :
> IDENT
> ;
> expression:
> additiveExpression
> ;
> // addition/subtraction
> additiveExpression:
> multiplicativeExpression((PLUS^ | MINUS^)
> multiplicativeExpression)*
> ;
> // multiplication/division
> multiplicativeExpression:
> powerExpression ( (STAR^ | DIV^ ) powerExpression )*
> ;
> powerExpression:
> unaryExpression ( POWER^ unaryExpression)*
> ;
> unaryExpression:
> MINUS^ unaryExpression
> | primaryExpression
> ;
> primaryExpression:
> NUMBER
> | LPAREN! additiveExpression RPAREN!
> ;
>
> class TestLexer extends Lexer;
> options {
> exportVocab=Test; // call the vocabulary "Java"
> }
>
> tokens {
> CONSTANT ="constant";
> }
>
> WS :( ' ' | '\t' | '\n' | '\r' )
> { _ttype = Token.SKIP; }
> ;
> //-------------
> // OPERATORS
> //-------------
> LPAREN : '(' ;
> RPAREN : ')' ;
> SEMI : ';' ;
> DOT : '.' ;
> DIV : '/' ;
> PLUS : '+' ;
> MINUS : '-' ;
> STAR : '*' ;
> POWER : '^' ;
>
> // Rule for IDENTIFIER: testLiterals is set to true.
> IDENT
> options {testLiterals=true;}
> : ( 'a'..'z' )+
> ;
>
> //for both integer and real number
>
> NUMBER : (DIGIT)+ ( '.' (DIGIT)+ )? ;
>
> //for numbers
> protected
> DIGIT : '0'..'9' ;
>
>
>
>
>
>
>
>
> Yahoo! Groups Links
>
>
>
>
>
>
>
> Yahoo! Groups Links
>
>
>
>
>
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
>
> This e-mail may contain information that is privileged, confidential or
> otherwise protected from disclosure. It must not be used by, or its
> contents copied or disclosed to, persons other than the intended
> recipient. However, the contents of this e-mail may be intercepted,
> monitored or recorded by Insurance Technology Solutions Limited for
> the purposes of ensuring compliance with its policies and procedures.
> Any liability (in negligence or otherwise) arising from any third party
> acting, or refraining from acting, on any information contained in this
> e-mail is excluded.
>
> Any views expressed in this message are those of the individual
> sender and do not necessarily represent the views of
> Insurance Technology Solutions Limited.
>
> If you have received this e-mail in error please notify the
> originator of the message.
>
> Scanning of this message and addition of this footer is performed
> by SurfControl E-mail Filter software in conjunction with
> virus detection software.
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/antlr-interest/
<*> To unsubscribe from this group, send an email to:
antlr-interest-unsubscribe at yahoogroups.com
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
More information about the antlr-interest
mailing list