[antlr-interest] nested parsing (BSDL)

[Mark Whitis]

Thanks for responding, see below.

On Sun, 30 Dec 2007, Harald M. Müller wrote:

> I admit I only scanned your email: But the "textbook approach", in my
> opinion would be the following:
>
> 1. Have a VHDL parser, which accepts strings simply as strings.
> 2. Do an AST rewrite, where you do "constant folding", i.e. you compute
> expressions that are made up only of constants. This will at least
> concatenate the &-separated strings.
> 3. Do another AST rewrite where you interpret the strings by calling another
> lexer+parser on them, return an AST from them and put that AST into the
> already existing AST structure.

Way too complicated for this application.   Something similar is done
where an existing VHDL tool is adapted.   There are no suitable
VHDL tools for adaptation in this case.

Although I think antlr should be able to create a clean AST tree from
this language (otherwise it suffers from an "omega inconsistency", a 
descrepency between the antlr language itself and the real world), I
probably won't use them.    The results of the parse can simply be
stored in structs, arrays, and combinations thereof.

> Another possibility would be to try to define a "LONG_STRING" symbol in the
> VHDL lexer, which allows string constants with & and comments to be passed
> to the lexer as a simple string; thus, you could get rid of stage 2. above;
> and maybe it is then possible to call the nested parsers of step 3.
> immediately while parsing, so that you end up with a 1-pass machine

Well, I was looking for suggestions on how to do something along these 
lines.   Given the documentation limitations, I can't even tell if antlr 
can do this, let alone how.

> (but I  would definitely keep the multiple grammars apart).

I really do not want to keep the grammars separate.

This is a BSDL parser, not a BSDL retrofit to VHDL.  BSDL was designed so 
you could do either.  I want to logically keep the grammar in something 
resembling the form it really should have been in the first place.  The 
whole "VHDL" subset thing is more of a bug than a feature, the result of 
design by committee.

> This possibility will not allow constructs with parentheses like
>
> constant DW_PACKAGE:PIN_MAP_STRING:="CLK:1, Q:(2,3,4,5,7,8,9,10), " &
>           ( "D:(23,22,21,20,19,17,16,15),"  &
>             ( "GND:6, VCC:18, OC_NEG:24, TDO:11, TMS:12, TCK:13, TDI:14" )
>           );
>
> which might be allowed in the syntax of VHDL. Also, with constant folding
> you could maybe tackle beasts like
>
>   constant B0 = "00000000";
>   constant B1 = "00000001";
>   constant B2 = "00000010";
>   constant B3 = "00000011";
>   ...
>   constant T0 = "10000000";
>   constant T1 = "10000001";
>   ...
>
>   attribute INSTRUCTION_OPCODE of ttl74bct8374 : entity is
>        "BYPASS (11111111, 10001000, 00000101, 10000100, 00000001),"   &
>        "EXTEST (" & B0 & ", " & T0 & "," &
>        "SAMPLE (" & B2 & ", " & T2 & "," &
>   ...
>
> or something like that where one starts to use constants in the "nested
> program."

Understand what you are getting at, however, this is prohibitted
in the language itself, fortunately.   Concatentation of litteral
strings is the only form of expression allowed.   Aside from
the string anarchy and complexity in the model itself, BSDL
is very rigid in a lot of ways about how things are expressed.
The order of statements, for example, is fixed.  If they had
simply been a little more rigid about a few things like where strings were 
broken, you wouldn't need a parser at all - a simple perl script
would do the job.

The design of BSDL could have been MUCH better if they had not made
the VHDL mistake and had included some other information.
But at least they tried to keep it simple.   BSDL is a descriptive
language, not a programming or synthesis one.   Exception being
IEEE STD 1532 programming macros which should have used a small
subset of VHDL coding features (or better yet, a tiny subset of C) but 
didn't.   Instead, you have something like three layers of programming
macros with no control flow.

VHDL would have made sense if vendors were providing behavioral
models of ICs, but they don't.   And even if that were the case,
this would not have been the way to do it.    Real extensions to
VHDL would have made sense; they would have been useful for more
than BSDL.   VHDL and Verilog is a flawed model that only represents a 
subset of the important information.  This is the kind of stuff you tend
to get from the expensive beaurocratic design by committee standards
organizations.

However, I think that parsing inside of strings has a lot of
applications besides BSDL and needs to be supported and documented.
Needing, for example, 20 different grammars because you have 20 different
string types, though, would leave a lot to be desired.