[antlr-interest] [C#] Lexer Not Throwing Errors
Justin Holewinski
justin.holewinski at gmail.com
Thu Apr 21 09:50:51 PDT 2011
Just a quick follow-up: an easy re-production is to use the Antlr-provided
examples-v3 archive. In the csharp/LL-star directory, change the SimpleC.g
file to use the CSharp3 target instead of CSharp2, add "System." to the
WriteLine methods, and use either the Antlr jar archive or the C# tool to
generate the grammar files. Upon testing, the provided "input" file works
as expected, but again, any other file is also parsed *without errors*.
Test run:
Good:
jholewinski at rapture [LL-star]$ mono Test.exe input
Processing file: /Volumes/Data/personal/examples-v3/csharp/LL-star/input
bar is a declaration
foo is a definition
Finished processing file:
/Volumes/Data/personal/examples-v3/csharp/LL-star/input
Bad:
jholewinski at rapture [LL-star]$ mono Test.exe output
Processing file: /Volumes/Data/personal/examples-v3/csharp/LL-star/output
Finished processing file:
/Volumes/Data/personal/examples-v3/csharp/LL-star/output
jholewinski at rapture [LL-star]$ mono Test.exe files
Processing file: /Volumes/Data/personal/examples-v3/csharp/LL-star/files
Finished processing file:
/Volumes/Data/personal/examples-v3/csharp/LL-star/files
jholewinski at rapture [LL-star]$ mono Test.exe Main.cs
Processing file: /Volumes/Data/personal/examples-v3/csharp/LL-star/Main.cs
Main is a definition
Finished processing file:
/Volumes/Data/personal/examples-v3/csharp/LL-star/Main.cs
jholewinski at rapture [LL-star]$
On Tue, Apr 19, 2011 at 12:49 PM, Justin Holewinski <
justin.holewinski at gmail.com> wrote:
> I forgot to add that if I instead use the Java target, I get lexer
> exceptions thrown on invalid input.
>
>
> On Tue, Apr 19, 2011 at 12:48 PM, Justin Holewinski <
> justin.holewinski at gmail.com> wrote:
>
>> I'm using the C# 3 port of Antlr (target and code generator) and I'm
>> experiencing a lack of lexer errors being generated. As a concrete example,
>> consider the following grammar:
>>
>> grammar CSTest;
>>
>> public
>> compilation_unit
>> : IDENTIFIER* EOF
>> ;
>>
>> IDENTIFIER
>> : (ID_PART) (ID_PART_OR_NUMBER)*
>> ;
>>
>> fragment
>> ID_PART
>> : ('a'..'z' | 'A'..'Z' | '_')
>> ;
>>
>> fragment
>> ID_PART_OR_NUMBER
>> : ID_PART
>> | '0'..'9'
>> ;
>>
>> WS
>> : (' '|'\r'|'\t'|'\n') { /*Skip();*/ $channel=Hidden; }
>> ;
>>
>> The only production in this grammar should allow for an arbitrary sequence
>> of identifiers, and it does. However, it also accepts ALL input, including
>> binary files and anything else I pipe into it. Is the default in the C#
>> target to silently ignore any lexing errors? In my driver program, I am not
>> seeing any throw exceptions when I call the compilation_unit rule.
>>
>> using System;
>> using System.IO;
>> using Antlr.Runtime;
>>
>>
>> namespace CSTest {
>> public class Program {
>> public static void Main(string[] args) {
>> string filename;
>> filename = args[0];
>> if(!Path.IsPathRooted(filename)) {
>> filename = Path.Combine(Environment.CurrentDirectory, filename);
>> }
>> ICharStream input = new ANTLRFileStream(filename);
>> CSTestLexer lexer = new CSTestLexer(input);
>> CommonTokenStream tokens = new CommonTokenStream(lexer);
>> CSTestParser parser = new CSTestParser(tokens);
>> try {
>> parser.compilation_unit();
>> }
>> catch(RecognitionException re) {
>> Console.Out.WriteLine("EXCEPTION");
>> Console.Out.WriteLine(re.StackTrace);
>> }
>>
>> }
>>
>> }
>>
>> }
>>
>> --
>>
>> Thanks,
>>
>> Justin Holewinski
>>
>>
>
>
> --
>
> Thanks,
>
> Justin Holewinski
>
>
--
Thanks,
Justin Holewinski
More information about the antlr-interest
mailing list