[antlr-interest] "An Introduction to ANTLR" presentation slides

[Micheal J]

I find this helpful:
    - Lexer - recognize word-level structure
    - Parser/TreeParser - recognize phrase-level structure (TreeParsers not
needed in phrase-level structure recognition for many (most?) programming
langauges)
 
All may also perform further [semantic] analysis so the whole system can
resolve context-dependent ambiguities (e.g. in the token stream
ID("start_process") LPAREN RPARENT SEMI that represents a function-call, is
"start_process" the name of a function and is the signature compatible).
 
Most such context-dependent analysis is done in the Parser/TreeParser where
there is much more context available. To recognize some languages (e.g.
C++), even the lexer is involved in resolving context-dependent ambiguities
(i.e. "is involved in semantic analysis").
 
Of course all three - lexer, parser, treeparser (particularly treeparsers) -
[can] do more than just recognize. They can also transform/translate what
they recognize...
 
Micheal
 



  _____  

From: antlr-interest-bounces at antlr.org
[mailto:antlr-interest-bounces at antlr.org] On Behalf Of Andy Tripp
Sent: 04 March 2008 01:03
To: Loring Craymer
Cc: antlr-interest; Terence Parr
Subject: Re: [antlr-interest] "An Introduction to ANTLR" presentation slides


Loring Craymer wrote: 

As the noted logician Charles Lutwidge Dodgson wrote
March Hare: .Then you should say what you mean. 
Alice: I do; at least - at least I mean what I say -- that's the same thing,
you know.
Hatter: Not the same thing a bit! Why, you might just as well say that, 'I
see what I eat' is the same as 'I eat what I see'!
March Hare: You might just as well say, that "I like what I get" is the same
thing as "I get what I like"! 
The Dormouse: You might just as well say, that "I breathe when I sleep" is
the same thing as "I sleep when I breathe"!

The same lexer that got you 'cat' will get you 'chat'.  

And the same person that got you "A Cat is a Feline" will get you "A chat is
a conversation". 
To the lexer, "cat" is the meaning of some input.


What meaning would you give that? 


Now suppose the sentence was "Viva la chat!".  Would your interpretation
change?


Not if I'm a lexer, no.
If I'm a parser, no.
If I'm a treealker, no.
In all three cases, "char" is still a word - some  output token passed from
a lexer to a parser.

As a person, of course. We have an entirely different architecture.

This example is a good illustration of why I keep harping on each phase
(lexer, parser, treewalker)
having it's own idea of what's valid input. And it doesn't make much sense
to me to
refer to all these different types of input as all "syntax". Suppose we have
two lexers (one for
ASCII only, one that handles unicode". And suppose we have two parsers, one
for English
sentences and another for French. And we have a single treewalker that
accepts 
AST input from either parser. Most people, including compiler writers, would
look
at a particular sentence and when asked "Is that correct syntax?", would
give just one
answer, not five. That says to me that "syntax" means something specific to
them, and
it doesn't mean "the form of any sort of input".



Lexers understand the character syntax from which words are built, not the
semantics of the words.


Well, no program really "understands" syntax or semantics.
It's just software manipulating symbols.

So, I'd reword that to say lexers *accept* certain input.
Here, I think sometimes people do talk about a lexer accepting certain
*syntax*, but more often,
they'll say it accepts a certain "set of characters" or something like that.

So, take the "understanding" parts out, and what do you have? 
Any software just takes input and produces output. 
I would call that process "assigning meaning". It certainly can be done
either with or without
"semantic analysis".



--Loring



-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20080304/58b681a1/attachment-0001.html