[antlr-interest] Problem with String parsing

preitz sharma p4rst2 at gmail.com
Tue Apr 19 22:45:10 PDT 2011


Hi,
I am facing problem in parsing String value.
Here is my grammar:

grammar stringProblem;


expr           :  SET attribute EOF;

attribute      :  ARRAY (SIZE)? Int
                    |  OUT(PUTSTR)? str
                    ;

str               :  (CHAR | DOT  | Int)+ ;

Int                :  '0'..'9'+;

SET            :  'set';

ARRAY       :  'array';

SIZE            :  's'('i'('z'('e')?)?)?;

OUT             :  'out';

PUTSTR      :  'p'('u'('t'('s'('t'('r')?)?)?)?)?;

fragment CHAR    :  ('a'..'z');

Space          :  (' ' | '\t' | '\r' | '\n') {$channel=HIDDEN;};

DOT           :  ('\U0000' .. '\UFFFF');



And the class to test it is:


import org.antlr.runtime.ANTLRStringStream;
import org.antlr.runtime.CommonTokenStream;
import org.antlr.runtime.RecognitionException;

public class Demo {

    public static void main(String[] args) throws RecognitionException {
        try {
            ANTLRStringStream in = new ANTLRStringStream("set outp 100z");
            stringProblemLexer lexer = new stringProblemLexer(in);
            CommonTokenStream tokens = new CommonTokenStream(lexer);
            stringProblemParser parser = new stringProblemParser(tokens);
            parser.expr();
        } catch (Exception e) {
            System.out.println(e.getMessage());
        }
    }
}

When I give the input sting as : "set outputstr 123zx3%", it is working
fine.
But when I am giving some input which matches any of the token, like "set
output 123arr5", then I am getting error like: "line 1:17 mismatched
character '5' expecting 'a'"

This is happening because other lexer rules like SET, ARRAY etc are
specified before CHAR. Hence it is giving more priority to them rather than
CHAR.
So whenever some character comes, it first tries to match to the tokens with
higher priority. But this should not be the expected behavior.

Please help me out. What should I do to make it work?

-- 
Regards
Preeti Sharma


More information about the antlr-interest mailing list