[antlr-interest] Unparseable?

[Jim Idle]

On Tue, 2008-09-09 at 14:37 -0700, titech at metaorb.com wrote:

> Is this just unparseable?  It is a piece of a DSN file output from Eagle PCB.
> The string_quote piece and the single-quoted string seem to be deal-breakers.
> I'm not currently attempting to incorporate the character passed in the string_quote
> into the parser.  Even when it parses successfully, it seems to be
> breaking the strings up into numerous tokens instead of one token per string.


First, remove all the string literals from your parser and make them
lexer rules.

Next, create a STRING_QUOTE token:

STRING_QUOTE : 'string_quote' SPACE+ .
    {
        // String quote is always first non whitespace character after
keyword
       //
      String s = $text;

      setText(s.subString(s.length()-1, s.length());
    }
;
Now your token contains the quote.

Now
string_quote : OPN! STRING_QUOTE^ CPN! ;


That's it.

Jim

> 
> Any advice would be greatly appreciated.
> 
> Thanks,
> --Mike
> 
> ==========================================================================
> 
> (PCB "C:/Archive/Development/EagleProjects/NewUSBToaster/NewUSBToaster_3.brd"
>   (parser
>     (string_quote ")
>     (space_in_quoted_tokens on)
>     (host_cad CadSoft)
>     (host_version 'EAGLE Version 4.16r2 Copyright (c) 1988-2006 CadSoft')
>   )
>   (structure
>     (layer "1#Top"  (type signal))
>     (layer "16#Bottom"  (type signal))
>   )
> )
> 
> ==========================================================================
> 
> grammar SQTest_11;
> 
> options
> {
>   output=AST;
>   ASTLabelType=CommonTree;
> }
> 
> header    : OPN!   'PCB'^ string_t dsnparser structure CPN!;
> 
> dsnparser : OPN! 'parser'^  string_quote
>                 space_in_quoted_tokens
>                 host_cad
>                 host_version CPN!; 
> 
> string_quote :   OPN  'string_quote'        '"'          CPN           -> ^('string_quote' '"')
>                | OPN  'string_quote'        '\''         CPN           -> ^('string_quote' '\'')
>                | OPN  'string_quote'        OTHER          CPN           -> ^('string_quote' OTHER);
> 
> space_in_quoted_tokens : OPN! 'space_in_quoted_tokens'^ bool_t CPN!;
> 
> host_cad : OPN! 'host_cad'^ id_t CPN!;
>     
> host_version : OPN! 'host_version'^ string_t? CPN!;
> 
> 
> structure : OPN! 'structure'^ 
>             layer*
>             CPN!; 
> 
> layer           :  OPN! 'layer'^ string_t type CPN!;
> 
> type            :  OPN! 'type'^ id_t CPN!;
>     
> string_t        :  sq_or_dq_string;
> 
> id_t            :  ID;
> 
> bool_t          :  true_or_false;
> 
> //Composite types
> true_or_false           :         TRUE | FALSE;
> 
> sq_or_dq_string            :         single_quoted_string | double_quoted_string;
> 
> //Basic types
> single_quoted_string    :         '\'' ( options {greedy=false;} : . )* '\'';
> 
> double_quoted_string    :         '"' ( options {greedy=false;} : . )* '"';
> 
> FLOAT    :       SIGN? ('0'..'9')+ '.' ('0'..'9')+;
> 
> INT    :       SIGN? ('0'..'9')+;
> 
> TRUE    :     'on' | 'true' | 'yes' | '1';
> 
> FALSE    :    'off' | 'false' | 'no' | '0';
> 
> ID    : (LETTER|'_')(LETTER|'_'|DIGIT)*;
> 
> fragment LETTER : LOWER | UPPER;
> fragment LOWER  : 'a'..'z';
> fragment UPPER  : 'A'..'Z';
> fragment DIGIT  : '0'..'9';
> fragment SPACE  : ' ' | '\t';
> fragment SIGN   : '+' | '-';
> NEWLINE            : ('\r'? '\n')+ { $channel = HIDDEN; };
> WHITESPACE      : SPACE+ { $channel = HIDDEN; };
> 
> OPN       : '(';
> CPN       : ')';
> OTHER    : .;
> 
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20080909/e51f8b9f/attachment.html