[antlr-interest] Adding a Space Leads to Mismatch
Quintin Beukes
quintin.beukes at signio.co.za
Fri Feb 10 00:17:21 PST 2012
I have tried to skip whitespace and have used tokens. The above
grammar is mostly just in debug state.
If I can narrow down the problem even further. The lexer keeps
predicting the "If " to be ALPHANUMERICSPACE, so the lexer fails. I
can actually not see why it would even do that, because this string
can never even match ALPHANUMERICSPACE.
Input:
(If )
grammar DebugA;
tokens {
IF = 'If';
OB = '(';
CB = ')';
}
fieldRules
: rule
EOF
;
rule
: OB ruleExpression CB NEWLINE
;
ruleExpression
: IF ALPHANUMERIC
;
ALPHANUMERIC
: ('a'..'z' | 'A'..'Z' | '0'..'9')+
;
ALPHANUMERICSPACE
: ('a'..'z' | 'A'..'Z' | '0'..'9')+ (' '+ ('a'..'z' | 'A'..'Z' | '0'..'9')+)*
;
WS
: (' '|'\t')+ {skip();}
;
NEWLINE
: '\r'? '\n'
;
Quintin Beukes
On Thu, Feb 9, 2012 at 9:30 PM, Jim Idle <jimi at temporal-wave.com> wrote:
> Don't use 'strings' in your parser, create real tokens and list the
> keywords and punctuation in the lexer before the generic rule. Also, it
> does not look like you need the spaces, so try skipping them:
>
> LPAREN: '(' ;
> ...
> KEYWORD: 'keyword';
> ....
> ALPHANUMERICSPACE: 'A'..'Z'+ ... etc
>
> WS: (' '|'\t')+ { skip(); } ; // Then remove WS refs in your parser
>
>
> Jim
>
>> -----Original Message-----
>> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
>> bounces at antlr.org] On Behalf Of Quintin Beukes
>> Sent: Thursday, February 09, 2012 11:20 AM
>> To: antlr-interest at antlr.org
>> Subject: Re: [antlr-interest] Adding a Space Leads to Mismatch
>>
>> I debugged the Lexer, and it seems that it's predictions for the next
>> token always seems to match against ALPHANUMERICSPACE.
>>
>> How can I resolve such a prediction error? Even if just pointing me to
>> the wiki.
>>
>> thanks,
>> Quintin Beukes
>>
>> List: http://www.antlr.org/mailman/listinfo/antlr-interest
>> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
>> email-address
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address
More information about the antlr-interest
mailing list