[antlr-interest] Antlr v4 - C++ target
Loring Craymer
lgcraymer at yahoo.com
Wed Jan 11 01:12:02 PST 2012
If Jim did not implement the vtable indirection (that could be easily changed, if so), then there is a little more opportunity for optimization, but still the problem is that state information takes up much more memory than does the text in tokens.
--Loring
>________________________________
> From: Ruslan Zasukhin <ruslan_zasukhin at valentina-db.com>
>To: Loring Craymer <lgcraymer at yahoo.com>; Richard Gildea <rgildea at gmail.com>; Sam Harwell <sam at tunnelvisionlabs.com>
>Cc: "antlr-interest at antlr.org" <antlr-interest at antlr.org>
>Sent: Wednesday, January 11, 2012 12:31 AM
>Subject: Re: [antlr-interest] Antlr v4 - C++ target
>
>On 1/11/12 5:36 AM, "Loring Craymer" <lgcraymer at yahoo.com> wrote:
>
>> I think a bit of caution is in order. The average text length of a token is 5
>> characters/bytes or less; pointers are now 8 bytes. Tokens encapsulate state
>> information as well as text--input source, character index, line number, token
>> type, C++ vtable pointers, and so forth--and are necessarily much larger than
>> the text they represent. Data items are aligned on word (8 byte) boundaries,
>> which adds even more expansion. It is possible to compress data structures by
>> removing alignment constraints, converting pointers to offsets (instead of an
>> end of text pointer, keep a byte or short integer index), and some other
>> tricks; these all cost in terms of performance. Further, there is the issue
>> of minimum allocation size; allocated memory chunks contain hidden pointers
>> and (depending on the compiler) are likely to be allocated in chunks that are
>> multiples of 128 bytes or larger. Shrinking ANTLR data structures, which are
>> already near-minimal in terms
>> of preserved information, is not easy to do. That is especially true for
>> C++; the reason that Jim Idle did a C target for ANTLR 3 in the first place
>> was to reduce the footprint of the ANTLR 2 C++ target.
>
>But Lorin,
>
>Again, I think C target have not achieve that point because was NOT single
>pointer to virtual table, but MANY pointers to all "class" methods.
>
>Or I am wrong here?
>
>
>> So--despite Sam's intentions, I would not expect the ANTLR 4 C++ memory
>> footprint to be smaller than the ANTLR 3 C footprint. If Sam does manage to
>> produce a small footprint target, feel free to applaud wildly. Just don't
>> expect miracles until they are made real.
>
>
>
>--
>Best regards,
>
>Ruslan Zasukhin
>VP Engineering and New Technology
>Paradigma Software, Inc
>
>Valentina - Joining Worlds of Information
>http://www.paradigmasoft.com
>
>[I feel the need: the need for speed]
>
>
>
>
>
More information about the antlr-interest
mailing list