[antlr-interest] More information about token location
forumer at smartmobili.com
forumer at smartmobili.com
Mon Jul 30 04:27:06 PDT 2012
Hi,
I am using antlr 3.4.1 with the c# backend to parse some java code and
I would like to get more information when
using the lexer. For the moment the IToken interface is defined like
that :
public interface IToken
{
int Channel { get; set; }
int CharPositionInLine { get; set; }
ICharStream InputStream { get; set; }
int Line { get; set; }
int StartIndex { get; set; }
int StopIndex { get; set; }
string Text { get; set; }
int TokenIndex { get; set; }
int Type { get; set; }
}
But when I have a token I would like to know where is the line where
the token starts but also where it ends (this
would be very handy for multiline comments for instance).
Would it be very difficult to modify the c# backend to get the
following interface :
public interface IToken
{
int Channel { get; set; }
int StartCharPositionInLine { get; set; }
int EndCharPositionInLine { get; set; }
ICharStream InputStream { get; set; }
int StartLine { get; set; }
int StopLine { get; set; }
int StartIndex { get; set; }
int StopIndex { get; set; }
string Text { get; set; }
int TokenIndex { get; set; }
int Type { get; set; }
}
Of course before using the lexer I could parse my entire buffer and
build a map between index range and line number
but I find it stupid to do it because it could be done during the
parsing.
More information about the antlr-interest
mailing list