[antlr-interest] Is there any way to tokenize using the delimiter in lexer.java?

[Joan Pujol]

Hi,

You only have to define the tokens well and the lexer will do the good job.
Your problem is in your lexer rules. If  you want post them.

Cheers,


On Tue, 14 Sep 2004 04:57:53 -0000, ganesh_rgk2003
<ganesh_rgk2003 at yahoo.com> wrote:
> In Lexer.java the input string is tokenized based on the tokens
> defined under this.
> 
> I want to tokenize giving my own delimiter.
> 
> eg:
> I have two tokens, say name and age defined in lexer.
> And also have one more token alphanumeric.
> 
> Now, if my input string is
> 
>  name=raj age123=21
> 
> I need to tokenize this by giving delimiter as blankspace and '=' and
> generate four tokens 'name', 'raj', 'age123' and '21'.
> 
> Instead, since name, age and alphanumeric are the tokens defined, my
> lexer is generating token as follows,
> 
>     1.    'name'
>     2.    '=raj'
>     3.    'age'
>     4.    '123=21'
> which i dont require.
> 
> IS THERE ANY WAY WHERE I CAN SPECIFY IN LEXER TO TOKENIZE ACCORDINGLY?
> 
> Thanks in advance.
> Ganz
> 
> 
> Yahoo! Groups Links
> 
> 
> 
> 
> 



-- 
Joan Jesús Pujol Espinar


 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
    antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/