[antlr-interest] Grammar EOL or END token

[Ramirez, Paul M (388J)]

Hi All,

I'm looking for some help with a grammar that has been written to parse a text file which contains some metadata. We are using the grammar to build up an object representation of the metadata file as the grammar is run. This is being done to run higher level validation and allow programmatic access to the metadata. The issue at hand is that the format has nesting mechanisms that when not closed as expected (i.e. EOF or an "END" token) we would like exit gracefully by closing off the nesting statement.

Here is an example of the format:

PDS_VERSION_ID = PDS3
LABEL_REVISION_NOTE = "some text here"

OBJECT = IMAGE
   LINES = 960
   LINE_SAMPLES = 956
   SAMPLE_TYPE = UNSIGNED_INTEGER
   SAMPLE_BITS = 8
   SAMPLE_BIT_MASK = 2#1111111111#
END_OBJECT

END

So what you can tell here is we really have a "key = value" type file with some simple nesting. Below are snippets of the grammar for this file. What we want is to be able to gracefully exit the "OBJECT = IMAGE" statement (rule) if we come upon the EOF or "END" token instead of the expected "END_OBJECT" token. I left in the code that we are using to generate the object model to give context but the real issue is what grammar changes to make.

label[Label label]
@init {this.label = label;}
@after {label.checkLineLengths();}
    : ( s=statement[label] {if (s != null) {label.addStatement(s);}} )*
      (END | EOF)
    ;

statement[Label label] returns [Statement result = null]
    : s=simple_statement[label]
        {result = s;}
    | g=group_statement[label]
        {result = g;}
    | o=object_statement[label]
        {result = o;}
    ;
    catch [RecognitionException re] {
        while(input.LA(1)!= EOL && input.LA(1) != CharStream.EOF) {
            input.consume();
        }
        if(input.LA(1) == EOL) {
            input.consume();
        }
        if (!suppress)
            label.addProblem(re.line, re.charPositionInLine, re.getMessage(), ProblemType.PARSE_ERROR);
    }

object_statement[Label label] returns [ObjectStatement result = null]
    : 'OBJECT' nl '=' nl id=IDENTIFIER (c=COMMENT)? EOL
        {
            result = new ObjectStatement(id.getLine(), id.getText(), label.getFile());
            if (c != null) {
                CommentStatement comment = new CommentStatement(c.getLine(), label.getFile());
                comment.setComment(c.getText());
                result.attachComment(comment);
            }
        }
      (
        s=statement[label] {if (s != null) {result.addStatement(s);}}
      |
        (~ END_OBJECT) => t=.
          {
              if (!suppress) {
                  label.addProblem(t.getLine(),t.getCharPositionInLine(), "Illegal start of statement: '" + t.getText() + "'", ProblemType.PARSE_ERROR);
              }
          }
        (~ EOL)* EOL
      )*
        END_OBJECT ('=' id2=IDENTIFIER)?
        (c2=COMMENT)? EOL
        {
            if (result != null && c2 != null) {
                CommentStatement comment = new CommentStatement(c2.getLine(), label.getFile());
                comment.setComment(c.getText());
                result.attachComment(comment);
            }
        }
    ;


Thanks, in advance for any help you can provide.

Sincerely,
Paul Ramirez
Jet Propulsion Laboratory
pramirez at jpl.nasa.gov

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20090916/a609aa2c/attachment.html