[antlr-interest] Context-Sensitive Follow Sets.. Bug?

[Andrey Desyatov]

Hi, Jim.

There is complete grammar that can reproducae the bug:

grammar HLSL;

options
{
	language = 'CSharp2';
}

fx_file :	variableSyntax+ ';';

variableSyntax
	:	storageClass* typeModifier* ident ident

	;

storageClass
	: 'extern' | 'nointerpolation' | 'precise' | 'shared' | 'groupshared'
| 'static' | 'uniform' | 'volatile'
	;
	
typeModifier
	: 'const' | 'row_major' | 'column_major'
	;

ident
: Identifier;


Identifier
	:	Letter (Letter|JavaIDDigit)*
	;

QuotedIdentifier
	:	'`' Identifier
	;

fragment
Letter
	:  '\u0024' |
		'\u0041'..'\u005a' |
		'\u005f' |
		'\u0061'..'\u007a' |
		'\u00c0'..'\u00d6' |
		'\u00d8'..'\u00f6' |
		'\u00f8'..'\u00ff' |
		'\u0100'..'\u1fff' |
		'\u3040'..'\u318f' |
		'\u3300'..'\u337f' |
		'\u3400'..'\u3d2d' |
		'\u4e00'..'\u9fff' |
		'\uf900'..'\ufaff'
	;

fragment
JavaIDDigit
	:  '\u0030'..'\u0039' |
		'\u0660'..'\u0669' |
		'\u06f0'..'\u06f9' |
		'\u0966'..'\u096f' |
		'\u09e6'..'\u09ef' |
		'\u0a66'..'\u0a6f' |
		'\u0ae6'..'\u0aef' |
		'\u0b66'..'\u0b6f' |
		'\u0be7'..'\u0bef' |
		'\u0c66'..'\u0c6f' |
		'\u0ce6'..'\u0cef' |
		'\u0d66'..'\u0d6f' |
		'\u0e50'..'\u0e59' |
		'\u0ed0'..'\u0ed9' |
		'\u1040'..'\u1049'
	;

WS  :  (' '|'\r'|'\t'|'\u000C'|'\n') {Skip();}
	;


I use the latest version and as I've understood Ter has already
admitted that the bug is presented
(http://www.antlr.org/pipermail/antlr-interest/2010-May/038783.html).
So the question is: is there any progress in fixing or any workaround
how to get the correct bit sets?

Kind regards
Andrey

2010/10/1 Jim Idle <jimi at temporal-wave.com>:
> You need to post a complete grammar, not just a few fragments - sorry. I
> also assume that you are using the latest version?
>
> Jim
>
>> -----Original Message-----
>> From: Andrey Desyatov [mailto:andrey.desyatov at gmail.com]
>> Sent: Friday, October 01, 2010 10:21 AM
>> To: Jim Idle
>> Cc: antlr-interest at antlr.org
>> Subject: Re: [antlr-interest] Context-Sensitive Follow Sets.. Bug?
>>
>> Thanks, Jim for fast reply.
>>
>> I can't give you any info about the grammar from May thread.
>> But in my grammar (the second link:
>> http://www.antlr.org/pipermail/antlr-interest/2010-
>> September/039764.html)
>> I can see the same behavior (described at the link location).
>> Definitions for Identifier, WS, Comments and so on are typical (according
> to
>> the book "The Definitive ANTLR Reference: Building domain-specific
>> languages").
>> Could you have a look on my grammar and the problem?
>>
>> Kind regards,
>> Andrey
>>
>> 2010/10/1 Jim Idle <jimi at temporal-wave.com>:
>> >
>> >
>> >> -----Original Message-----
>> >> From: Andrey Desyatov [mailto:andrey.desyatov at gmail.com]
>> >> Sent: Friday, October 01, 2010 9:28 AM
>> >> To: Jim Idle
>> >> Cc: antlr-interest at antlr.org
>> >> Subject: Re: [antlr-interest] Context-Sensitive Follow Sets.. Bug?
>> >>
>> >> Sorry, Jim for the confusing question.
>> >> I've tried to continue thread from May with subject
>> >> "Context-Sensitive Follow Sets.. Bug?"
>> >> (http://www.antlr.org/pipermail/antlr-interest/2010-May/038776.html),
>> >> but without success (it was my second post to mailing list and I
>> >> couldn't understand how to continue any thread).
>> >
>> > I had to remove the many stars from the grammar to see what it was:
>> >
>> > start: animal (AND acClass)? service EOF;
>> >
>> > animal: (DOG | CAT );
>> >
>> > service: (HARDWARE | SOFTWARE) ;
>> >
>> > AND: 'and';
>> >
>> > DOG: 'dog';
>> >
>> > CAT: 'cat';
>> >
>> > HARDWARE: 'hardware';
>> >
>> >
>> > SOFTWARE: 'software';
>> >
>> > acClass
>> > @init*
>> > {
>> >
>> > System.out.println(computeContextSensitiveRuleFOLLOW().toString());
>> > }
>> > :
>> > ;
>> >
>> > And the OP says:
>> >
>> >
>> > Testing this grammar, with let's say input:
>> > "*dog and software*",
>> > the result in the console is:
>> > "{4, 7, 8}" (which stands for tokens {THE, HARDWARE, SOFTWARE}),
>> >
>> > However, the grammar given does not have a token for THE, or a
>> > specification for WS, or a grammar xzy;
>> >
>> > so the grammar is incomplete. I suspect that there is a problem with
>> > the grammar and not the followsets. If someone can give a complete
>> > example, then I can look at it
>> >
>> > Jim
>> >
>> >
>> >
>> >>
>> >> Description of my problem I've given in another thread:
>> >> http://www.antlr.org/pipermail/antlr-interest/2010-
>> >> September/039764.html.
>> >>
>> >
>> >
>> >
>> > List: http://www.antlr.org/mailman/listinfo/antlr-interest
>> > Unsubscribe:
>> > http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>> >
>
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>



-- 
С уважением,
Десятов Андрей