[antlr-interest] [C#] Lexer Not Throwing Errors

Justin Holewinski justin.holewinski at gmail.com
Tue Apr 19 09:48:43 PDT 2011


I'm using the C# 3 port of Antlr (target and code generator) and I'm
experiencing a lack of lexer errors being generated.  As a concrete example,
consider the following grammar:

grammar CSTest;

public
compilation_unit
  : IDENTIFIER* EOF
  ;

IDENTIFIER
  : (ID_PART) (ID_PART_OR_NUMBER)*
  ;

fragment
ID_PART
  : ('a'..'z' | 'A'..'Z' | '_')
  ;

fragment
ID_PART_OR_NUMBER
  : ID_PART
  | '0'..'9'
  ;

WS
  :  (' '|'\r'|'\t'|'\n') { /*Skip();*/ $channel=Hidden; }
  ;

The only production in this grammar should allow for an arbitrary sequence
of identifiers, and it does.  However, it also accepts ALL input, including
binary files and anything else I pipe into it.  Is the default in the C#
target to silently ignore any lexing errors?  In my driver program, I am not
seeing any throw exceptions when I call the compilation_unit rule.

using System;
using System.IO;
using Antlr.Runtime;


namespace CSTest {
  public class Program {
    public static void Main(string[] args) {
      string filename;
      filename = args[0];
      if(!Path.IsPathRooted(filename)) {
        filename = Path.Combine(Environment.CurrentDirectory, filename);
      }
      ICharStream input = new ANTLRFileStream(filename);
      CSTestLexer lexer = new CSTestLexer(input);
      CommonTokenStream tokens = new CommonTokenStream(lexer);
      CSTestParser parser = new CSTestParser(tokens);
try {
parser.compilation_unit();
}
catch(RecognitionException re) {
  Console.Out.WriteLine("EXCEPTION");
Console.Out.WriteLine(re.StackTrace);
}

    }

  }

}

-- 

Thanks,

Justin Holewinski


More information about the antlr-interest mailing list