[antlr-interest] ANTLR 2.7.7 - prepend to InputBuffer
Alex Marin
alex.marin at amiq.ro
Wed Aug 18 07:59:59 PDT 2010
Hello everyone,
Is it possible in ANTLR 2.7.7 to add characters on the fly to a lexer's
InputBuffer? Do you have any suggestion/guideline/experience on how to
do this? (without re-implementing the InputBuffer/CharQueue classes)
I will explain the reason behind the need to do this. The language we
are lexing allows pre-processing, by using C-like defines (or macros).
// Define foo macro
`define foo this is the replacement of foo
// Use foo macro
`foo // this expands to 'this is the replacement of foo'
// Define moo macro with a parameter
`define moo(x) x+2
// Use moo macro
`moo(1) // this expands to '1+2'
At the moment, when our lexer encounters a macro, it starts another
lexer which lexes the replacement. E.g:
...
`define foo this is the replacement of foo
...
`foo // Here another lexer is started and provided the input stream
'this is the replacement of foo'
...
This approach works very well at the time being, except for the
situations when the replacement text is not a fully "lexable" piece of
code, like in the example below:
...
`define a(x,y) x+y // Define macro a with parameters x and y
`define b `a(1 // Define macro b without parameters
...
`b,2) // Use macro b; this should expand to `a(1,2) and therefore to 1+2
...
Now, the problem is that the lexer we start for `b will lex '`a(`' which
is not lexically correct and will fail with an error.
The best solution would be to be able to insert in the main lexer's
buffer the replacement of `b and continue lexing normally. Of course, no
new lexer would be needed then. So, after matching `b, the buffer would
look like `a(1,2)
Thanks in advance,
Alex
More information about the antlr-interest
mailing list