[antlr-interest] Theading Tree Walkers

[Loring Craymer]

C++ garbage collection (uses reference counting in ASTRef) for ANTLR 2 is _not_ thread safe, nor was it intended to be.  You'll have to judiciously add in lock/unlock calls to make it so.  AFAIK, no one has taken the effort to make any of the ANTLR runtimes thread safe, nor is there any reason to do so under normal conditions--ANTLR operates serially on input data, including trees.

--Loring



----- Original Message ----
> From: "code_dude at nym.hush.com" <code_dude at nym.hush.com>
> To: antlr-interest at antlr.org
> Sent: Monday, February 2, 2009 9:11:30 AM
> Subject: [antlr-interest] Theading Tree Walkers
> 
> Hi there,
> Im using antr-2.7.6 (C++ code generation) and have a threading 
> problem with the antlr generated tree walker. 
> 
> Basically what Im doing if taking the parse tree and sorting the 
> expressions ( using my custom function openmp_sort(tr, &inn_vtr) ) 
> into  their own basic blocks then and then parallel process (using 
> Open MP) the expressions in each block.  
> 
> The thing I when I run it , every so often I get a segmentation 
> fault from my tree parser. Below is the stack trace from ddd
>   
> 
> antlr::ASTRef::~ASTRef ()  (destructor called from ASTRefCount.hpp)
> ncoTree::assign            (process an essign expression)  
> ncoTree::out               (expression walker)
> ncoTree::statements        (statement walker) 
> 
> 
> Below is an example of my "openmp_sort(RefAST 
> tr,std::vector&inn_vtr)"  function in action. Also 
> attached is ( a big simplification ) code detailing how I invoke 
> the antlr generated code. I hope I have made clear what I'm doing.
> 
> 
> 
> // Input Expressions
> A=10;
> B=20;
> C=30;
> D=40;
> A++;
> B*=20;
> C-=40;
> E=A+B;
> F=A+B+C+D+E;
> G=A;
> G++;
> D++;
> H=20/++A;
> I=A+C+D;
> J=40;
> K=H*(C-=1);
> 
> // Sorted Expressions
> -------------------------------
> ( EXPR ( = A 10 ) )
> ( EXPR ( = B 20 ) )
> ( EXPR ( = C 30 ) )
> ( EXPR ( = D 40 ) )
> ( EXPR ( = J 40 ) )
> -------------------------------
> -------------------------------
> ( EXPR ( POST_INC A ) )
> ( EXPR ( *= B 20 ) )
> ( EXPR ( -= C 40 ) )
> -------------------------------
> -------------------------------
> ( EXPR ( = E ( + A B ) ) )
> ( EXPR ( = G A ) )
> -------------------------------
> -------------------------------
> ( EXPR ( = F ( + ( + ( + ( + A B ) C ) D ) E ) ) )
> ( EXPR ( POST_INC G ) )
> -------------------------------
> -------------------------------
> ( EXPR ( POST_INC D ) )
> ( EXPR ( = H ( / 20 ( ++ A ) ) ) )
> -------------------------------
> -------------------------------
> ( EXPR ( = I ( + ( + A C ) D ) ) )
> -------------------------------
> -------------------------------
> ( EXPR ( = K ( * H ( -= C 1 ) ) ) )
> -------------------------------
> 
> 
> /********************** Code 
> *************************************************/
> 
> int parse_antlr(std::vector&prs_vtr,char* fl_spt_usr,char 
> *cmd_ln_sng)
> {
>   
>   ANTLR_USING_NAMESPACE(std);
>   ANTLR_USING_NAMESPACE(antlr);
>   
> 
>   int idx;  
>   prs_cls *prs_arg;
>   std::string filename(fl_spt_usr);
> 
>   std::vectorwlk_vtr;
>   std::vectorinn_vtr;
> 
>   ncoLexer *lexer=NULL;
>   ncoParser *parser=NULL;
> 
>   istringstream *sin=NULL;
>   ifstream *in=NULL;
>   
>   
>   RefAST tr,a;
>   ASTFactory ast_factory;
> 
>   prs_arg=&prs_vtr[0]; 
>   
>   std::vectorwlk_vtr;
>   std::vectorinn_vtr;
> 
> 
>     
>     if( cmd_ln_sng ){
>       sin= new  istringstream(cmd_ln_sng);
>       lexer= new ncoLexer( *sin, prs_arg);
>       selector.addInputStream(lexer,cmd_ln_sng);
>       selector.select(cmd_ln_sng);
>     }else {
>       in=new ifstream(filename.c_str());          
>       lexer= new ncoLexer( *in, prs_arg);
>       selector.addInputStream(lexer,filename);
>       selector.select(filename);
> 
>     }    
>     
>     
>     lexer->setFilename(filename);
>     
>     parser= new ncoParser(selector);
>     parser->setFilename(filename);
>     parser->inc_vtr.push_back(filename);
>     
> 
>     parser->initializeASTFactory(ast_factory);
>     parser->setASTFactory(&ast_factory);
>     
>     
>     // Parse the input expressions
>     parser->program();
>     a = parser->getAST();
>   
>   tr=a;
> 
>   // Do an OpenMP Sort
>   (void)openmp_sort(tr,&inn_vtr);  
>   
>   // initialize walkers  
>     ncoTree* wlk_obj;    
>     for(idx=0 ; idx< (int)prs_vtr.size(); idx++){
>       wlk_obj=new ncoTree(&prs_vtr[idx]);  
>       wlk_obj->initializeASTFactory(ast_factory);
>       wlk_obj->setASTFactory(&ast_factory);
>       wlk_vtr.push_back(wlk_obj); 
>     }      
>     
>     
> 
> 
>     // walk the tree in parallel processing statements
>   
> #ifdef _OPENMP
> #pragma omp parallel for default(none) private(kdx,wlk_vtr,tr) 
> #endif
>    for(kdx=0 ;kdx< nbr_sz; kdx++) {      
> 
>      tr=inn_vtr[kdx];
>      wlk_vtr[omp_get_thread_num]->statements(tr);
> 
> 
>    } //end OPENMP parallel loop
> 
> 
> 
> 
> 
> 
>   
>   return 1;
> }
> 
> 
> /********************** Code 
> *************************************************/
> 
> 
> 
> 
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: 
> http://www.antlr.org/mailman/options/antlr-interest/your-email-address