[antlr-interest] Theading Tree Walkers
code_dude at nym.hush.com
code_dude at nym.hush.com
Mon Feb 2 11:09:34 PST 2009
Thanks Loring,
Thats a bit of a blow- My interpreter depends heavily on Tree
parsing, and it needs to be multi-threaded.
Can you think of any quick fixes ?
Would the correct solution to be convert the Parse tree into a
thread safe tree Of my choosing ?
what about creating my Parse tree with a custom ast factory thats
thread safe ?
Regards code dude
On Mon, 02 Feb 2009 18:51:14 +0000 Loring Craymer
<lgcraymer at yahoo.com> wrote:
>C++ garbage collection (uses reference counting in ASTRef) for
>ANTLR 2 is _not_ thread safe, nor was it intended to be. You'll
>have to judiciously add in lock/unlock calls to make it so.
>AFAIK, no one has taken the effort to make any of the ANTLR
>runtimes thread safe, nor is there any reason to do so under
>normal conditions--ANTLR operates serially on input data,
>including trees.
>
>--Loring
>
>
>
>----- Original Message ----
>> From: "code_dude at nym.hush.com" <code_dude at nym.hush.com>
>> To: antlr-interest at antlr.org
>> Sent: Monday, February 2, 2009 9:11:30 AM
>> Subject: [antlr-interest] Theading Tree Walkers
>>
>> Hi there,
>> Im using antr-2.7.6 (C++ code generation) and have a threading
>> problem with the antlr generated tree walker.
>>
>> Basically what Im doing if taking the parse tree and sorting the
>
>> expressions ( using my custom function openmp_sort(tr, &inn_vtr)
>)
>> into their own basic blocks then and then parallel process
>(using
>> Open MP) the expressions in each block.
>>
>> The thing I when I run it , every so often I get a segmentation
>> fault from my tree parser. Below is the stack trace from ddd
>>
>>
>> antlr::ASTRef::~ASTRef () (destructor called from
>ASTRefCount.hpp)
>> ncoTree::assign (process an essign expression)
>> ncoTree::out (expression walker)
>> ncoTree::statements (statement walker)
>>
>>
>> Below is an example of my "openmp_sort(RefAST
>> tr,std::vector&inn_vtr)" function in action. Also
>> attached is ( a big simplification ) code detailing how I invoke
>
>> the antlr generated code. I hope I have made clear what I'm
>doing.
>>
>>
>>
>> // Input Expressions
>> A=10;
>> B=20;
>> C=30;
>> D=40;
>> A++;
>> B*=20;
>> C-=40;
>> E=A+B;
>> F=A+B+C+D+E;
>> G=A;
>> G++;
>> D++;
>> H=20/++A;
>> I=A+C+D;
>> J=40;
>> K=H*(C-=1);
>>
>> // Sorted Expressions
>> -------------------------------
>> ( EXPR ( = A 10 ) )
>> ( EXPR ( = B 20 ) )
>> ( EXPR ( = C 30 ) )
>> ( EXPR ( = D 40 ) )
>> ( EXPR ( = J 40 ) )
>> -------------------------------
>> -------------------------------
>> ( EXPR ( POST_INC A ) )
>> ( EXPR ( *= B 20 ) )
>> ( EXPR ( -= C 40 ) )
>> -------------------------------
>> -------------------------------
>> ( EXPR ( = E ( + A B ) ) )
>> ( EXPR ( = G A ) )
>> -------------------------------
>> -------------------------------
>> ( EXPR ( = F ( + ( + ( + ( + A B ) C ) D ) E ) ) )
>> ( EXPR ( POST_INC G ) )
>> -------------------------------
>> -------------------------------
>> ( EXPR ( POST_INC D ) )
>> ( EXPR ( = H ( / 20 ( ++ A ) ) ) )
>> -------------------------------
>> -------------------------------
>> ( EXPR ( = I ( + ( + A C ) D ) ) )
>> -------------------------------
>> -------------------------------
>> ( EXPR ( = K ( * H ( -= C 1 ) ) ) )
>> -------------------------------
>>
>>
>> /********************** Code
>> *************************************************/
>>
>> int parse_antlr(std::vector&prs_vtr,char* fl_spt_usr,char
>> *cmd_ln_sng)
>> {
>>
>> ANTLR_USING_NAMESPACE(std);
>> ANTLR_USING_NAMESPACE(antlr);
>>
>>
>> int idx;
>> prs_cls *prs_arg;
>> std::string filename(fl_spt_usr);
>>
>> std::vectorwlk_vtr;
>> std::vectorinn_vtr;
>>
>> ncoLexer *lexer=NULL;
>> ncoParser *parser=NULL;
>>
>> istringstream *sin=NULL;
>> ifstream *in=NULL;
>>
>>
>> RefAST tr,a;
>> ASTFactory ast_factory;
>>
>> prs_arg=&prs_vtr[0];
>>
>> std::vectorwlk_vtr;
>> std::vectorinn_vtr;
>>
>>
>>
>> if( cmd_ln_sng ){
>> sin= new istringstream(cmd_ln_sng);
>> lexer= new ncoLexer( *sin, prs_arg);
>> selector.addInputStream(lexer,cmd_ln_sng);
>> selector.select(cmd_ln_sng);
>> }else {
>> in=new ifstream(filename.c_str());
>> lexer= new ncoLexer( *in, prs_arg);
>> selector.addInputStream(lexer,filename);
>> selector.select(filename);
>>
>> }
>>
>>
>> lexer->setFilename(filename);
>>
>> parser= new ncoParser(selector);
>> parser->setFilename(filename);
>> parser->inc_vtr.push_back(filename);
>>
>>
>> parser->initializeASTFactory(ast_factory);
>> parser->setASTFactory(&ast_factory);
>>
>>
>> // Parse the input expressions
>> parser->program();
>> a = parser->getAST();
>>
>> tr=a;
>>
>> // Do an OpenMP Sort
>> (void)openmp_sort(tr,&inn_vtr);
>>
>> // initialize walkers
>> ncoTree* wlk_obj;
>> for(idx=0 ; idx< (int)prs_vtr.size(); idx++){
>> wlk_obj=new ncoTree(&prs_vtr[idx]);
>> wlk_obj->initializeASTFactory(ast_factory);
>> wlk_obj->setASTFactory(&ast_factory);
>> wlk_vtr.push_back(wlk_obj);
>> }
>>
>>
>>
>>
>> // walk the tree in parallel processing statements
>>
>> #ifdef _OPENMP
>> #pragma omp parallel for default(none) private(kdx,wlk_vtr,tr)
>> #endif
>> for(kdx=0 ;kdx< nbr_sz; kdx++) {
>>
>> tr=inn_vtr[kdx];
>> wlk_vtr[omp_get_thread_num]->statements(tr);
>>
>>
>> } //end OPENMP parallel loop
>>
>>
>>
>>
>>
>>
>>
>> return 1;
>> }
>>
>>
>> /********************** Code
>> *************************************************/
>>
>>
>>
>>
>> List: http://www.antlr.org/mailman/listinfo/antlr-interest
>> Unsubscribe:
>> http://www.antlr.org/mailman/options/antlr-interest/your-email-
>address
More information about the antlr-interest
mailing list