[antlr-interest] Quoted Strings in ANTLR 3.0.1 and 3.1.1

Fromm, Stefan Fromm at dresden-informatik.de
Fri Oct 24 09:17:15 PDT 2008


Hello Sam,

thanks for giving sense to  the spent time :)

Stefan

> -----Ursprüngliche Nachricht-----
> Von: Sam Harwell [mailto:sharwell at pixelminegames.com] 
> Gesendet: Freitag, 24. Oktober 2008 18:15
> An: Fromm, Stefan; antlr-interest at antlr.org
> Betreff: RE: [antlr-interest] Quoted Strings in ANTLR 3.0.1 and 3.1.1
> 
> No time wasted, now we have a great set of test cases for 
> fixing the interpreter bugs. :) Good to hear it's resolved for you.
> 
> Sam
> 
> -----Original Message-----
> From: antlr-interest-bounces at antlr.org 
> [mailto:antlr-interest-bounces at antlr.org] On Behalf Of Fromm, Stefan
> Sent: Friday, October 24, 2008 11:13 AM
> To: antlr-interest at antlr.org
> Subject: Re: [antlr-interest] Quoted Strings in ANTLR 3.0.1 and 3.1.1
> 
> Hello,
> 
> that's right. I thought that the interpreter delivers 
> reliable results. Sorry for wasting time. I have tested it 
> with the debugger and in fact the test cases are working.
> 
> What for is the interpreter in ANTLRworks?
> 
> Regards
> Stefan
> 
> 
> 
> > -----Ursprüngliche Nachricht-----
> > Von: Sam Harwell [mailto:sharwell at pixelminegames.com]
> > Gesendet: Freitag, 24. Oktober 2008 09:08
> > An: Fromm, Stefan; antlr-interest at antlr.org
> > Betreff: RE: [antlr-interest] Quoted Strings in ANTLR 3.0.1 
> and 3.1.1
> > 
> > You're running it in the interpreter, not the debugger. The actual 
> > generated code works across the board with all three 
> versions of the 
> > grammar below.
> > 
> > Sam
> > 
> > -----Original Message-----
> > From: antlr-interest-bounces at antlr.org 
> > [mailto:antlr-interest-bounces at antlr.org] On Behalf Of Fromm, Stefan
> > Sent: Friday, October 24, 2008 1:00 AM
> > To: antlr-interest at antlr.org
> > Subject: Re: [antlr-interest] Quoted Strings in ANTLR 3.0.1 
> and 3.1.1
> > 
> >  I tested it again with ANTLR version 3.0.1, 3.1 and 3.1.1. 
> > Furthermore I added tests with a slightly changed lexer rule for 
> > quoted strings.
> > Here are my results.
> > 
> > grammar QuotedStringTest1;
> > 
> > QUOTED_STRING
> >    : '\'' ( '\'\'' | ~('\'') )* '\''
> >    ;
> > 	
> > NUMBER
> >   :	 ('+' | '-')? '0'..'9'+ ('.' '0'..'9'+)?
> >   ;
> > 
> > value_expr
> >   :	 (QUOTED_STRING | NUMBER) EOF
> >   ;
> > 
> > Pattern         ANTLR 3.0.1                  ANTLR 3.1
> > ANTLR 3.1.1
> >                 ANTLRworks 1.1.7             ANTLRworks 1.2
> > ANTLRworks 1.2.1
> > --------------------------------------------------------------
> > ----------
> > ---------------------------
> > ''              matched as expected          matched as expected
> > matched as expected
> > ''''            matched as expected          matched as expected
> > matched as expected
> > ''''''          matched as expected          matched as expected
> > matched as expected
> > '''a'           matched as expected          matched as expected
> > matched as expected
> > '''a'''         matched as expected          matched as expected
> > matched as expected
> > '''a''b'        matched as expected          matched as expected
> > matched as expected
> > '''a''b'''      matched as expected          matched as expected
> > matched as expected
> >  
> > 
> > 'a'''           matched as expected          no match, 
> expected match
> > no match, expected match
> > 'a''b'          matched as expected          matched 
> partly, only 'a''
> > matched partly, only 'a''
> > 'a''b''c'       matched as expected          no match, 
> expected match
> > no match, expected match
> > 
> > 
> > 
> > grammar QuotedStringTest2;
> > 
> > QUOTED_STRING
> >    : '\'' ( ~('\'') | '\'\'')* '\''
> >    ;
> > 	
> > NUMBER
> >   :	 ('+' | '-')? '0'..'9'+ ('.' '0'..'9'+)?
> >   ;
> > 
> > value_expr
> >   :	 (QUOTED_STRING | NUMBER) EOF
> >   ;
> > 
> > Pattern         ANTLR 3.0.1                  ANTLR 3.1
> > ANTLR 3.1.1
> >                 ANTLRworks 1.1.7             ANTLRworks 1.2
> > ANTLRworks 1.2.1
> > --------------------------------------------------------------
> > ----------
> > ---------------------------
> > ''              matched as expected          matched as expected
> > matched as expected
> > ''''            matched as expected          no match, 
> expected match
> > no match, expected match
> > ''''''          matched as expected          no match, 
> expected match
> > no match, expected match
> > '''a'           matched as expected          no match, 
> expected match
> > no match, expected match
> > '''a'''         matched as expected          no match, 
> expected match
> > no match, expected match
> > '''a''b'        matched as expected          no match, 
> expected match
> > no match, expected match
> > '''a''b'''      matched as expected          no match, 
> expected match
> > no match, expected match
> >                 
> > 'a'''           matched as expected          matched as expected
> > matched as expected
> > 'a''b'          matched as expected          matched as expected
> > matched as expected
> > 'a''b''c'       matched as expected          matched as expected
> > matched as expected
> > 
> > Interestingly changing the order of ('\'\'' | ~('\'')) to (~('\'') |
> > '\'\'') makes exactly the opposite test cases fail in 3.1 and 3.1.1.
> > First variant lets pass all expressions where first escaped 
> quotes ''
> > come before any letter. Second variant lets pass all 
> expressions where 
> > first escaped quotes come after any letter.
> > 
> > 
> > 
> > I have also tried the grammar of Sam with all three versions:
> > 
> > grammar TestQuotedString;
> > 
> > value
> > 	:	(	QUOTED_STRING
> > 		|	NUMBER
> > 		)*
> > 		EOF
> > 	;
> > 
> > QUOTED_STRING
> > 	:	'\'' ( '\'\'' | ~('\'') )* '\''
> > 	;
> > 
> > NUMBER
> > 	:	('+' | '-')?
> > 		('0'..'9')+
> > 		('.' ('0'..'9')+)?
> > 	;
> > 
> > 
> > 
> > Pattern         ANTLR 3.0.1                  ANTLR 3.1
> > ANTLR 3.1.1
> >                 ANTLRworks 1.1.7             ANTLRworks 1.2
> > ANTLRworks 1.2.1
> > --------------------------------------------------------------
> > ----------
> > ---------------------------
> > ''              matched as expected          matched as expected
> > matched as expected
> > ''''            matched as expected          matched as expected
> > matched as expected
> > ''''''          matched as expected          matched as expected
> > matched as expected
> > '''a'           matched as expected          matched as expected
> > matched as expected
> > '''a'''         matched as expected          matched as expected
> > matched as expected
> > '''a''b'        matched as expected          matched as expected
> > matched as expected
> > '''a''b'''      matched as expected          matched as expected
> > matched as expected
> >                 
> > 'a'''           matched as expected          matched only EOF
> > matched only EOF
> > 'a''b'          matched as expected          matched only 'a''
> > matched only 'a''
> > 'a''b''c'       matched as expected          matched 'a'' and 'b''
> > matched 'a'' and 'b''
> > 
> > The long test
> > '''a'0'''a'''0'''a''b'''0'a'''0'a''b'0'a''b''c'0''0''''0''''''
> > 0'''a''b'
> > matched exactly as expected in all three versions. In fact Sams 
> > grammar would not be usable for my use case, because SQL does not 
> > allow mixing of quoted strings/numbers in value expressions.
> > 
> > Stefan
> > 
> > List: http://www.antlr.org/mailman/listinfo/antlr-interest
> > Unsubscribe:
> > http://www.antlr.org/mailman/options/antlr-interest/your-
> > email-addr
> > ess
> > 
> > 
> 
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: 
> http://www.antlr.org/mailman/options/antlr-interest/your-
> email-address
> 
> 


More information about the antlr-interest mailing list