[antlr-interest] token context information
tcurdt
tcurdt at yahoo.com
Wed Jun 16 02:46:36 PDT 2004
This is maybe a pretty general question but...
I've run into the following problem. I have
a file with key value pairs like this:
key1 : value1
key2 : value2
Now what I'd like have is a token stream
of
KEY COLON VALUE
KEY COLON VALUE
but the KEY and the VALUE tokens are pretty
much the same. Except that the KEY token is
more restricted on particular chars. So the
only way I got this working was to have it
LITERAL COLON LITERAL
LITERAL COLON LITERAL
Which is bad and gives me some headaches due
to the character restrictions in the key.
If the lexer would have the context information
as the parser has he would "know" that after
COLON there is supposed to be a VALUE and only
needs to test that. But I don't know how to do
that with a grammar.
I also tried with fixed KEY tokens. So all
KEYs are known by the grammar. But as soon
as I write:
key1 : key1
The lexer recognizes the KEY1 token twice
KEY1 COLON KEY1
But I still wanna have
KEY1 COLON VALUE
What am I missing here?
Any pointers appreciated!
TIA
--
Torsten
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/antlr-interest/
<*> To unsubscribe from this group, send an email to:
antlr-interest-unsubscribe at yahoogroups.com
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
More information about the antlr-interest
mailing list