[antlr-interest] Antlr syntax reference

[Raphael Reitzig]

Hi Sam!

I am not sure what you mean by "Antlr's lexical analyser recognises that it
denotes a newline". Do you mean "ANTLR recognizes '\n' where I have a line
break in my input"?

If so, this is nothing special. In any ASCII file, which is a linear list
of characters of 8 Bit each, a line break is encoded by a special symbol.
For Unix, this is '\n'. So, your text editor puts a '\n' in your text if
you type ENTER and shows it to you as a line break, hiding existence of
'\n' from you. Thus, ANTLR _really_ finds the character '\n' in your input
and does nothing but following your rules.

Note that for different OS' the linebreak character is different. I. e.,
for Unix it's '\n' (or LF, line feed), whereas Mac uses '\r' (CR, carriage
return). Windows uses both at once. Refer to
http://en.wikipedia.org/wiki/Linebreak for mor details. To get your grammar
working for inputs created on either system, you may want to encode a
linebreak in ANTLR with a token rule like
LINEBREAK : '\n' | '\r' |'\r\n';

In general, I strongly recommend Terence Parr's book about ANTLR. It may
not adress this special issue, but it explains most (all?) aspects of ANTLR
in a entertaining way.

I hope I did not misunderstand your question.

Raphael

On Tue, 24 Jun 2008 15:12:16 +0100, "Sam Kuper" <sam.kuper at uclmail.net>
wrote:
> Dear all,
> 
> I am looking for an exhaustive guide to Antlr 3.0.1 syntax (I am using
> AntlrWorks 1.1.7); I'll explain why. My grammar so far looks like this:
> 
> grammar DCP;
> options {
>     language=Python;
> }
> dcp     : DOCUMENT* EOF;
> DOCUMENT    : HEADERS;
> HEADERS    : YEAR_HEADER MONTH_HEADER ;
> YEAR_HEADER    : '*Y 18\n';
> MONTH_HEADER    : '*M October\n';
> 
> Notice that in this grammar, I have used \n to denote new lines. But
> although I have not declared what \n is, Antlr's lexical analyser
> recognises
> that it denotes a newline; in other words, \n is a pre-defined token in
> Antlr grammar. I'm guessing there are others, and I want to be conscious
> of
> them as I work, but I have so far been unable to find a document that
> lists
> all the pre-defined tokens in Antlr's grammar. Presumably one exists
> somewhere. If you know where, please could you tell me?
> 
> Many thanks,
> 
> Sam