[antlr-interest] (newbie) very basic grammar for simple text and integer

ali azimi aliaazimi at yahoo.com
Sun Aug 5 20:40:31 PDT 2007


Hi,
   
  Thank you very much for your advice. I appreciate it alot.Could you also advise me for the following?
   
  I need to change my grammar slightly. I need to make my grammar understand to recognise two integers and parenthesis and comma in this form:
   
  ( INT , INT ). I need my parser create different tokens for ‘(‘ ‘)’, comma and INTs. 
   
  For this purpose, the suggested grammar will not work properly. Since, both AlphaNumeric and Integer rules use Decimaldigit rule. As a result the parser uses the Text rule to parse something like ( INT , INT ), so I will not have different tokens for INTs which I want.(I eventually need to extract the INTs from AST)In another word the parser will parse something like ( 2 , 4 ) not like ‘(‘ ‘2’ ‘,’ ‘4’ ‘)’ but like ‘( 2 , 4 )’ as one token (as a text). How can I tell the parser to use this rule: LPAR  INT COMMA INT RPAR to parse something like (2,4) not  the rule Text?
   
  And also with the suggested grammar, a simple text like following is parsed and put in three different nodes, How can I tell the parser to put all the text in one token so that I will have one node for whole the sentence not three.
   
  SIGNAL
  Newgame,Probe,Result,
  Endgame,Win,Lose,Score(Integer),Bump;
   
  Best regards,
   
  Al


Johannes Luber <jaluber at gmx.de> wrote:   ali azimi wrote:
> Hi,
> 
> Thank you so much for replying to me. I have tried the indstructions,
> however the problem still remains. I am simply not able to make the
> following grammar understand to parse a simple text or integer. When
> being debugged, and inputed some simple text or integer, the grammar
> emits the "MismatchedTokenException" error message.
> 
> Could you please help me?
> 
> The grammar is:
> 
> text :Text;
> integer :Integer;
> 
> Text :(AlphaNumeric|Special|Space|Apostrophe)* ;
> Integer :Decimaldigit+ ;
> 
> fragment Apostrophe:'\'';
> fragment Space :(' ')*;
> fragment Word : '.'* AlphaNumeric ( AlphaNumeric | '.' )* ;
> fragment CHARACTERSTRING : '\'' ( options{greedy=false;}:
> (~('\''|'\r'|'\n')| '\'' '\''))* '\'';
> fragment Special 
> :'+'|'-'|'!'|'/'|'>'|'8'|'('|')'|'"'|','|';'|'<'|'='|':'|'?'|'&'|'%'|'.'|'_'; 
> fragment AlphaNumeric :Uppercase|National|Lowercase|Decimaldigit;
> fragment Decimaldigit :'0'..'9' ;
> fragment National :'#'|'@'|'"'|'$'|'['|']'|'{'|'}'|'^'|'~' ;
> fragment Lowercase :'a'..'z' ;
> fragment Uppercase :'A'..'Z' ;
> 
> NEWLINE:'\r' ? '\n' {skip();};
> WS : (' ' |'\t' |'\n' |'\r' )+ {skip();} ;
> 
> I am very grateful.
> 
> Best regard,
> 
> Al
>

I've removed from Special all doubles of National. Furthermore I turned
all * into +, if they allowed the lexer rule to be empty. I'd also
advise to remove from Special ',' if you plan to separate the tokens
with commas. As it stands the WS rule is useless for spaces, as they end
as text tokens. You may want to exclude space as an allowed character
for the first position in Text. Also Word is unused, although I changed
it into a more correct version, if I gathered your intent correctly.

Best regards,
Johannes Luber


input_data : (Text|Integer)*;

Text :(AlphaNumeric|Special|Space|Apostrophe)+ ;
Integer :Decimaldigit+ ;

fragment Apostrophe:'\'';
fragment Space :' ';
fragment Word : ( AlphaNumeric | '.' )+ ;
fragment CHARACTERSTRING : '\'' ( options{greedy=false;}:
(~('\''|'\r'|'\n')| '\'' '\''))* '\'';
fragment Special
:'+'|'-'|'!'|'/'|'>'|'('|')'|','|';'|'<'|'='|':'|'?'|'&'|'%'|'.'|'_';
fragment AlphaNumeric :Uppercase|National|Lowercase|Decimaldigit;
fragment Decimaldigit :'0'..'9' ;
fragment National :'#'|'@'|'\"'|'$'|'['|']'|'{'|'}'|'^'|'~' ;
fragment Lowercase :'a'..'z' ;
fragment Uppercase :'A'..'Z' ;

NEWLINE:'\r' ? '\n' {skip();};
WS : (' ' |'\t' |'\n' |'\r' )+ {skip();} ;


       
---------------------------------
Yahoo! oneSearch: Finally,  mobile search that gives answers, not web links. 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20070805/337c5290/attachment.html 


More information about the antlr-interest mailing list