[antlr-interest] [BUG] 3.0b4 no complaint on parser reference to lexical fragment

John B. Brodie jbb at acm.org
Sun Nov 12 20:31:28 PST 2006


>The key point to see here is that the parser does not "call" a lexer rule!

Of course.      ...where, in this thread, was it stated otherwise.

>When I stretch my mind a bit, I can even imagine that I'd actually want to
>emit tokens for fragment rules.

Of course. fragment rules are inside the lexer and so may call emit() as an
action, just like any other lexer rule. And of course if any lexer rule wants
to emit() multiple tokens it can do so, but the default lexer only buffers 1
token at a time so must be modified. See the lexer example i posted to this
list (after Dr. Parr helped me) that emits multiple tokens when trying to
differentiate between integers, reals and the range ".."  operator.

>what have you won? you might get around fiddling with the token's text's in
>the parser, you could possibly set up a finer control of token channels, etc.
>This might be a bad idea, but is interesting nonetheless. ;)

who is talking about any of that in this thread?




There is an interface between a Parser and a Lexer. The Lexer produces a
stream of Tokens which the Parser consumes.

And of what type should these lexer produced Tokens be?

Should that type include the tokens{}, lexer rules *AND* fragment rules?

Or should that type be restricted *AND ENFORCED* to just tokens{} and
lexer rules --- with fragments excluded.

>I need coffee. Quick.

I need sleep. Good Night.
   -jbb


More information about the antlr-interest mailing list