[antlr-interest] Antlr v4 - C++ target

Loring Craymer lgcraymer at yahoo.com
Wed Jan 11 01:12:02 PST 2012


If Jim did not implement the vtable indirection (that could be easily changed, if so), then there is a little more opportunity for optimization, but still the problem is that state information takes up much more memory than does the text in tokens.

--Loring



>________________________________
> From: Ruslan Zasukhin <ruslan_zasukhin at valentina-db.com>
>To: Loring Craymer <lgcraymer at yahoo.com>; Richard Gildea <rgildea at gmail.com>; Sam Harwell <sam at tunnelvisionlabs.com> 
>Cc: "antlr-interest at antlr.org" <antlr-interest at antlr.org> 
>Sent: Wednesday, January 11, 2012 12:31 AM
>Subject: Re: [antlr-interest] Antlr v4 - C++ target
> 
>On 1/11/12 5:36 AM, "Loring Craymer" <lgcraymer at yahoo.com> wrote:
>
>> I think a bit of caution is in order.  The average text length of a token is 5
>> characters/bytes or less; pointers are now 8 bytes.  Tokens encapsulate state
>> information as well as text--input source, character index, line number, token
>> type, C++ vtable pointers, and so forth--and are necessarily much larger than
>> the text they represent.  Data items are aligned on word (8 byte) boundaries,
>> which adds even more expansion.  It is possible to compress data structures by
>> removing alignment constraints, converting pointers to offsets (instead of an
>> end of text pointer, keep a byte or short integer index), and some other
>> tricks; these all cost in terms of performance.  Further, there is the issue
>> of minimum allocation size; allocated memory chunks contain hidden pointers
>> and (depending on the compiler) are likely to be allocated in chunks that are
>> multiples of 128 bytes or larger.  Shrinking ANTLR data structures, which are
>> already near-minimal in terms
>>  of preserved information, is not easy to do.  That is especially true for
>> C++; the reason that Jim Idle did a C target for ANTLR 3 in the first place
>> was to reduce the footprint of the ANTLR 2 C++ target.
>
>But Lorin, 
>
>Again, I think C target have not achieve that point because was NOT single
>pointer to virtual table, but MANY pointers to all "class" methods.
>
>Or I am wrong here?
>
>
>> So--despite Sam's intentions, I would not expect the ANTLR 4 C++ memory
>> footprint to be smaller than the ANTLR 3 C footprint.  If Sam does manage to
>> produce a small footprint target, feel free to applaud wildly.  Just don't
>> expect miracles until they are made real.
>
>
>
>-- 
>Best regards,
>
>Ruslan Zasukhin
>VP Engineering and New Technology
>Paradigma Software, Inc
>
>Valentina - Joining Worlds of Information
>http://www.paradigmasoft.com
>
>[I feel the need: the need for speed]
>
>
>
>
>


More information about the antlr-interest mailing list