[antlr-interest] recombining tokens
Davyd Madeley
davyd at fugro-fsi.com.au
Mon Jul 28 01:03:15 PDT 2008
Hi all,
I'm currently writing a grammar in which '/' is used to append a
qualifier to a token. Unfortunately it is also used in path parameters.
I am trying to figure out how I can recombine tokens in the case where I
determine I'm reading a path.
e.g.
// these are my token delimiters
TOKEN
: ~(','|'>'|'*'|'/'|'('|')'|LINE_TERMINATOR)+
;
At one point in the state machine, I expect to be able to start reading
parameters ('LINE' is a special token at the start of the file, but
after that is just a regular token):
parameter
: a=TOKEN -> PARAMETER[$a]
| a='LINE' -> PARAMETER[$a]
| path -> ^(PATH path)
;
path
: ('/' TOKEN)+
;
Every so often, a path will be provided. Currently this will be
tokenised around the '/', which is undesirable.
e.g.
PATH (9) .......................... PATH
'/' (20) ........................ /
TOKEN (11) ...................... path
'/' (20) ........................ /
TOKEN (11) ...................... to
'/' (20) ........................ /
TOKEN (11) ...................... my.file
What I want to do is be able to recombine this into a
PARAMETER["/path/to/my.file"].
Someone spoke about a concatenation operator, but I can't find any info
about it.
Regards,
--davyd
--
Davyd Madeley Software Engineer
Fugro Seismic Imaging, Perth Australia
More information about the antlr-interest
mailing list