[antlr-interest] Re: Unified grammar and # directives for a C -lik e language

[mzukowski at yci.com]

Set a flag indicating whether the character is the first on a line.
Something like this in your lexer:

  boolean lineStart=true;

  protected Token makeToken(int t)
  {
    if ( t != Token.SKIP) {
        lineStart=false;
    }
   super.makeToken();
  }

  public void newline() { 
        super.newline();
	  lineStart=true;
    }

Now your predicate will be {lineStart}?

Monty

-----Original Message-----
From: uprightness_of_character [mailto:andrei at metalanguage.com]
Sent: Wednesday, May 14, 2003 3:59 PM
To: antlr-interest at yahoogroups.com
Subject: [antlr-interest] Re: Unified grammar and # directives for a
C-lik e language


--- In antlr-interest at yahoogroups.com, mzukowski at y... wrote:
> You can now have predicates hoisted into nextToken for non-protected 
lexer
> rules.  This means you can write something like this:
> 
> HASH_DIRECTIVE_BEGIN: {getColumn()==1}? '#' ;

Thanks all for the answers. I was aware of this feature, however I 
can't (nicely) express what I want. 

The rule above does recognize a '#' directive starting in the first 
column. However, it fails to recognize a '#' directive that follows a 
space character. E.g., the directive in the second line below won't be 
recognized (and it should):

#include "a"
 #include "b"
#include "c"

Jim O'Connor suggested:

HASH_DIRECTIVE_BEGIN : {getColumn()==1}? ( WS )? '#' ;

(BTW, in my grammar WS is:

// Whitespace (no newlines) -- ignored
WS    :
    (
        ' '
        | '\t'
        | '\f'
    )+
    { $setType(antlr::Token::SKIP); }
    ;

)

So now I get the following ambiguity:

warning:lexical nondeterminism between rules WS and 
HASH_DIRECTIVE_BEGIN upon
ril.g:     k==1:' '
ril.g:     k==2:<end-of-token>,' '
ril.g:     k==3:<end-of-token>,' '

This grammar will still recognize '#' only if it appears in column 1. 
What solution would allow spaces before '#'?


Andrei


 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ 


 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/