[antlr-interest] Yet an another TokenStream ( for C++ Target )
Gavin Lambert
antlr at mirality.co.nz
Wed Feb 22 00:01:49 PST 2012
At 09:12 19/02/2012, Gokulakannan Somasundaram wrote:
>>> b) Every time a rule's action is executed in execution mode(
>>> when backtracking == 0 ), it will delete all the tokens
except
>>> the first and last token.
[...]
>If a user attempts to look ahead, there won't be any issue. If a
>user looks backward, it might result i an exception. If the user
>wants to access any token, it has to be attempted only from
>within the rule. This is the discipline a parser programmer has
>to follow to reduce the memory footprint. After the rule's scope
>is completed, the tokens might get deleted. So, the necessary
>info has to be stored in some variable at rule scope / global
>scope and accessed from there.
Why not have an option that wraps all Token references inside a
std::shared_ptr (or std::tr1::shared_ptr or boost::shared_ptr for
backwards compatibility)? That way the action code can cache
particular tokens for as long as required, while uninteresting
tokens (even start/stop tokens if the action code clears them) can
get automatically deleted as they fall out of scope.
(It shouldn't be the default though, as shared_ptrs aren't cheap.)
--
Gavin Lambert, Mirality Systems
<http://www.mirality.co.nz/>
<http://lambert.geek.nz/>
----
Anyone who goes to a psychiatrist ought to have their head
examined.
Few women admit their age. Few men act theirs.
More information about the antlr-interest
mailing list