[antlr-interest] Re: Short circuit of the lexer

[Terence Parr]

On Saturday, January 18, 2003, at 05:06 AM, xadeck 
<decoret at graphics.lcs.mit.edu> wrote:

> --- In antlr-interest at yahoogroups.com, Terence Parr <parrt at j...> wrote:
>>
>>
>>
>> Hi.  First, can you give us the simple rules that matched the array?
>> There may be a more efficient way to specify the language.
>
> The grammar is a complex stuff. I extracted the part for arrays:
> I guess values rule could be write as (INT)* but then I don't know how
> to push the integers read into my result vector. I don't want to build
> an AST unless really necessary (some files I parse have 180 000 lines
> and the AST may result in high memory cost).

Are you using the latest 2.7.2 stuff or 2.7.1?  I think 2.7.2 is faster 
:)

Also, (INT)* is definitely more efficient than the tail recursion you 
are using.  just add the action within the loop:

( i:INT {result.push_back(atoi(i->getText().c_str()));} )*

Put that in rule decl instead of referring to values and you should be 
good to go.  Let me know if this works.  The tail recursion will build 
a HUGE stack of method invocation records if you have 180k lines...very 
very inefficient.  Try the loop :)

Ter

> options {
>     language="Cpp";
> }
> {
> #include <deque>
>
> using namespace std;
>
> static vector<int> result;
>
> }
> class MyParser extends Parser;
> options
> {
>     k=2;
> }
> file:  (decl)*
>     ;
>
> decl : Id LBRACKET values RBRACKET
>     ;
>
> values: i1:INT
>         {
>             result.push_back(atoi(i1->getText().c_str()));
>         }
>     | i2:INT
>         {
>             result.push_back(atoi(i2->getText().c_str()));
>         }
>         values
>     ;
> class MyLexer extends Lexer;
> options
> {
>     k=2;
> }
> {
>     protected:
>     bool parsingList;
> }
> LBRACKET : '[';
> RBRACKET : ']';
>
> WS  :   (' '
>     |   '\t'
>     |   '\n'
>     |   '\r')
>         { $setType(ANTLR_USE_NAMESPACE(antlr)Token::SKIP); }
>     ;
>
> INT :   ('0'..'9')+
>     ;
>
> Id : ('a'..'z')+
>     ;
>
>
>
>> Terence
>> --
>> Co-founder, http://www.jguru.com
>> Creator, ANTLR Parser Generator: http://www.antlr.org
>> Lecturer in Comp. Sci., University of San Francisco
>
>
>
>
> Your use of Yahoo! Groups is subject to 
> http://docs.yahoo.com/info/terms/
>
>
--
Co-founder, http://www.jguru.com
Creator, ANTLR Parser Generator: http://www.antlr.org
Lecturer in Comp. Sci., University of San Francisco


 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/