[antlr-interest] How can I parse length defined tokens?
Dave Dutcher
dave at tridecap.com
Wed Dec 2 11:36:42 PST 2009
You have to remember that the lexer "automatically" chooses which token to
parse next. It doesn't know anything about your block_1 grammar rule, so
when it sees 10 digits in a row, it won't know to lex a session number
before a sequence number. I believe if the lexer has two rules with the
same left prefix it will try to lex the one which matches the most
characters.
(http://www.antlr.org/wiki/pages/viewpage.action?pageId=4882470)
I think your best option is to make application_id, service_id, address,
session_number, and sequence number grammar rules.
Dave
_____
From: antlr-interest-bounces at antlr.org
[mailto:antlr-interest-bounces at antlr.org] On Behalf Of
loic.lefevre at bnpparibas.com
Sent: Wednesday, December 02, 2009 11:15 AM
To: antlr-interest at antlr.org
Subject: [antlr-interest] How can I parse length defined tokens?
Hello,
I'm relatively new to antlr v3.
I'm trying to write a parser that should be able to parse the following
(SWIFT block 1) input:
{1:F01PARBDEFFXXXX0001000000}
I wish the parser to be able to then work with the following rule:
block_1: '{1:'
APPLICATION_ID
SERVICE_ID
ADDRESS
SESSION_NUMBER
SEQUENCE_NUMBER
'}'
;
with tokens:
APPLICATION_ID: 'F' | 'A' | 'L';
SERVICE_ID: '01' | '02';
fragment CHAR: 'A' .. 'Z' | '0' .. '9';
ADDRESS: CHAR CHAR CHAR CHAR CHAR CHAR CHAR CHAR CHAR CHAR CHAR CHAR;
fragment DIGIT: '0' .. '1';
SESSION_NUMBER: DIGIT DIGIT DIGIT DIGIT;
SEQUENCE_NUMBER: DIGIT DIGIT DIGIT DIGIT DIGIT DIGIT;
However it doesn't work. Do you know an elegant way to work with this kind
of
tokens? I mean to define length defined tokens and to be able to work
without
"token recognition overlapp".
(I'm open to combined grammar or not solution)
Could the "filter" Lexer option help me there?
Thanks in advance.
Regards,
Loïc
This message and any attachments (the "message") is
intended solely for the addressees and is confidential.
If you receive this message in error, please delete it and
immediately notify the sender. Any use not in accord with
its purpose, any dissemination or disclosure, either whole
or partial, is prohibited except formal approval. The internet
can not guarantee the integrity of this message.
BNP PARIBAS (and its subsidiaries) shall (will) not
therefore be liable for the message if modified.
Do not print this message unless it is necessary,
consider the environment.
---------------------------------------------
Ce message et toutes les pieces jointes (ci-apres le
"message") sont etablis a l'intention exclusive de ses
destinataires et sont confidentiels. Si vous recevez ce
message par erreur, merci de le detruire et d'en avertir
immediatement l'expediteur. Toute utilisation de ce
message non conforme a sa destination, toute diffusion
ou toute publication, totale ou partielle, est interdite, sauf
autorisation expresse. L'internet ne permettant pas
d'assurer l'integrite de ce message, BNP PARIBAS (et ses
filiales) decline(nt) toute responsabilite au titre de ce
message, dans l'hypothese ou il aurait ete modifie.
N'imprimez ce message que si necessaire,
pensez a l'environnement.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20091202/43f1b222/attachment.html
More information about the antlr-interest
mailing list