[antlr-interest] [C#] Lexer Not Throwing Errors
Justin Holewinski
justin.holewinski at gmail.com
Thu Apr 21 10:48:37 PDT 2011
On Thu, Apr 21, 2011 at 1:06 PM, Sam Harwell <sharwell at pixelminegames.com>wrote:
> Hi Justin,
>
> The exception does get thrown, but is handled in Lexer.NextToken() by
> tossing out a character and continuing. There are many ways to deal with
> this situation ranging from overriding NextToken in your lexer to creating
> an ANYCHAR rule at the very end that allows the incorrect input characters
> to pass on to the parser for handling there.
>
> ANYCHAR : . ;
>
I see. Out of curiosity, any reason why this is different from the bahavior
of the Java target?
>
> Sam
>
> -----Original Message-----
> From: antlr-interest-bounces at antlr.org
> [mailto:antlr-interest-bounces at antlr.org] On Behalf Of Justin Holewinski
> Sent: Tuesday, April 19, 2011 11:49 AM
> To: antlr-interest at antlr.org
> Subject: [antlr-interest] [C#] Lexer Not Throwing Errors
>
> I'm using the C# 3 port of Antlr (target and code generator) and I'm
> experiencing a lack of lexer errors being generated. As a concrete
> example,
> consider the following grammar:
>
> grammar CSTest;
>
> public
> compilation_unit
> : IDENTIFIER* EOF
> ;
>
> IDENTIFIER
> : (ID_PART) (ID_PART_OR_NUMBER)*
> ;
>
> fragment
> ID_PART
> : ('a'..'z' | 'A'..'Z' | '_')
> ;
>
> fragment
> ID_PART_OR_NUMBER
> : ID_PART
> | '0'..'9'
> ;
>
> WS
> : (' '|'\r'|'\t'|'\n') { /*Skip();*/ $channel=Hidden; }
> ;
>
> The only production in this grammar should allow for an arbitrary sequence
> of identifiers, and it does. However, it also accepts ALL input, including
> binary files and anything else I pipe into it. Is the default in the C#
> target to silently ignore any lexing errors? In my driver program, I am
> not
> seeing any throw exceptions when I call the compilation_unit rule.
>
> using System;
> using System.IO;
> using Antlr.Runtime;
>
>
> namespace CSTest {
> public class Program {
> public static void Main(string[] args) {
> string filename;
> filename = args[0];
> if(!Path.IsPathRooted(filename)) {
> filename = Path.Combine(Environment.CurrentDirectory, filename);
> }
> ICharStream input = new ANTLRFileStream(filename);
> CSTestLexer lexer = new CSTestLexer(input);
> CommonTokenStream tokens = new CommonTokenStream(lexer);
> CSTestParser parser = new CSTestParser(tokens); try {
> parser.compilation_unit(); } catch(RecognitionException re) {
> Console.Out.WriteLine("EXCEPTION");
> Console.Out.WriteLine(re.StackTrace);
> }
>
> }
>
> }
>
> }
>
> --
>
> Thanks,
>
> Justin Holewinski
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe:
> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>
>
--
Thanks,
Justin Holewinski
More information about the antlr-interest
mailing list