[antlr-interest] Is there any way to tokenize using the delimiter in lexer.java?
ganesh_rgk2003
ganesh_rgk2003 at yahoo.com
Mon Sep 13 21:57:53 PDT 2004
In Lexer.java the input string is tokenized based on the tokens
defined under this.
I want to tokenize giving my own delimiter.
eg:
I have two tokens, say name and age defined in lexer.
And also have one more token alphanumeric.
Now, if my input string is
name=raj age123=21
I need to tokenize this by giving delimiter as blankspace and '=' and
generate four tokens 'name', 'raj', 'age123' and '21'.
Instead, since name, age and alphanumeric are the tokens defined, my
lexer is generating token as follows,
1. 'name'
2. '=raj'
3. 'age'
4. '123=21'
which i dont require.
IS THERE ANY WAY WHERE I CAN SPECIFY IN LEXER TO TOKENIZE ACCORDINGLY?
Thanks in advance.
Ganz
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/antlr-interest/
<*> To unsubscribe from this group, send an email to:
antlr-interest-unsubscribe at yahoogroups.com
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
More information about the antlr-interest
mailing list