[antlr-interest] "An Introduction to ANTLR" presentation slides

[Gerald B. Rosenberg]

At 01:08 PM 2/29/2008, Andy Tripp wrote:
>Gerald B. Rosenberg wrote:
>>Even if they were represented as the same sub-tree, whether it 
>>could be unrolled or further transformed in some other way could be 
>>dependent on surrounding syntax.  Or, for that matter, dependent on 
>>the internal syntax of the X sub-tree.
>
>>So, it would be entirely valid for a syntactic predicate to 
>>consider the surrounding or internal syntax of a sub-tree in 
>>deciding on a "correct" production, even in a treewalker.
>Ah... I see...you're casually referring to AST structure as "syntax" 
>to give Terrence something convenient to site
>and show that he's not the only one who uses the word "syntax" that way ;)

I wish, but no.  The concept of syntax as structure was initially 
established long ago in the language sciences.  Formal statements of 
what "syntax" is abound, though a useful summary definition is, in 
effect, some structure of well-defined elements, whether they be 
words, hieroglyphics, or even sounds, used to communicate meaning by 
the represented structure.  In the same vein, semantics is, in 
effect, the inferences that can be drawn from syntax fragments, a 
cultural or use context, etc. that serve to refine, even 
disambiguate, the meaning of particular syntax.

The language sciences definition of syntax was merely adopted by 
computer scientists.  I don't think that Ter has or intends to use 
syntax in any different manner.

So, in the context of a formal constructional grammar, the formal 
syntax basis of a lexer is a stream of characters or, more 
accurately, atoms, which may be simple characters, Unicode 
codepoints, bytes or any other discrete unit of data.  For a parser, 
the formal syntax basis is a stream of tokens.  For a tree-walker, an 
ordered collection of nodes.  A constructional grammar merely defines 
a formal transformation from one syntax basis to another.  Of course, 
through each transformation, the full depth of the data structure can 
be carried forward by the implementation.

The concepts of syntax and semantics are orthogonal to any operative 
distinctions between a lexer, parser, and treeparser: all forms of 
semantic and syntactic predicates are equally valid in the lexer, 
parser, and tree grammars.

I do believe that you could peg syntax = literals and develop your 
own set of operative definitions.  But, other than satisfying a 
preconception that syntax = literals, don't see that it improves the 
conceptual view.  While history alone is not a satisfying reason to 
do anything, it is often a really good starting point.