[antlr-interest] BOF or BOL (Beginning of file/line)

Indhu Bharathi indhu.b at s7software.com
Sun May 31 18:33:11 PDT 2009


Overwride the emit method and check 'getCharPositionInLine' for the tokens 
emited. If the value is zero emit an additional BOL token. This will take 
care of beginning of file too.

Something like this:


java.util.LinkedList<Token> tokenQueue = new java.util.LinkedList<Token>();

public void emit(Token token) {

    // If this token is starting from column zero, emit a BOL token
    if( token.getCharPositionInLine()==0 ) {
        tokenQueue.offer(new ClassicToken(BOL, ""));
    }

    // Emit the actual token
    tokenQueue.offer(token);
    state.token = token;
}

public Token nextToken() {
    super.nextToken();
    if ( tokenQueue.size()==0 ) {
        return Token.EOF_TOKEN;
    }
    return (Token)tokenQueue.poll();
}


- Indhu

----- Original Message ----- 
From: "Warren Falk" <warren-antlr.org at warrenfalk.com>
To: <antlr-interest at antlr.org>
Sent: Monday, June 01, 2009 6:09 AM
Subject: [antlr-interest] BOF or BOL (Beginning of file/line)


For a grammar I'm working on, I need the lexer to output a token for
the beginning of (almost) every line. Adding an emit() to a custom
NEWLINE rule is simple enough, but what is the correct way to emit the
token at the beginning of the file?

Thanks in advance

List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address 



More information about the antlr-interest mailing list