[antlr-interest] Problems with Pre-processing instructions of C#

[Jim Idle]

On 09/20/2009 10:23 AM, Eduard Ralph wrote:
>
> Hi community,
>
> I'm fighting with the processing of pre-processing instructions 
> according to C# specs. The BNF is:
>
> Whitespace(opt) '#' Whitespace(opt) 'error' input-characters
>
> Whitespace(opt) '#' Whitespace(opt) 'warning' input-characters
>
> Whitespace(opt) '#' Whitespace(opt) 'line'  ...
>
> where
>
>  Whitespace(opt) can be optionally one or more spaces 
> ('\u0020','\u00A0', and a few more)
>
>  Input-characters is anything except newline ('\n', and a few more)
>
> I wrote in the Lexer, where the other rules are fragments
>
> PP_DIAGNOSTIC      :        (WHITESPACE* HASH WHITESPACE* 
> 'error')=>WHITESPACE* HASH WHITESPACE* ERROR INPUT_CHARACTER*
>
>                             |        (WHITESPACE* HASH WHITESPACE* 
> 'warning')=>WHITESPACE* HASH WHITESPACE* WARNING INPUT_CHARACTER*
>
>                             ;
>
> PP_LINE                 :        (WHITESPACE* HASH WHITESPACE* 
> 'line')=> WHITESPACE* HASH WHITESPACE* LINE PP_LINE_INDICATOR NEWLINE
>
>                             ;
>
> fragment PP_LINE_INDICATOR      :        INTEGER_LITERAL PP_FILE_NAME?
>
>                                                |        
> IDENTIFIER_OR_KEYWORD
>
>                                                ;
>
> fragment PP_FILE_NAME              :        STRING_LITERAL
>
>                                                ;
>
> fragment HASH                          :        '#';
>
> Unfortunately the DFA in the Lexer doesn't recognize correctly which 
> Lexer rule applies and now I'm wondering why this is so. Some thoughts 
> and help would be greatly appreciated.
>
>

You cannot do it like this. The pre-processor has to be in the lexer, as 
per the specs, but it is very tricky to get correct. Also you have to 
make it recursive and recognize the difference between the usages of 
'#'. It is more than can be explained in an email or two.

I have a working C# 3.x parser and tree parser, which you can try online 
here: 
http://www.temporal-wave.com/index.php?option=com_psrrun&view=psrrun&Itemid=58 
if you are interested.

It will save you a lot of time and already deals with the pre-processor 
(preserves in own channel) and every example in the spec, as well as 
compiling the mono source and mono regression tests (as a test), LINQ 
and the full spec basically.

Jim
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20090920/f6ca7572/attachment.html