[antlr-interest] Re: identifier with space

[jbb at acm.org]

>The problem is, with this grammer the string
>
>"SELECT a field with name FROM aTable"
>
>will be cut in 2 tokens:
>'SELECT'
>'a field with name FROM aTable'
>
>where as I want 4:
>'SELECT'
>'a field with name'
>'FROM'
>'aTable'
>
>> I am sure I must be missing something obvious but why not use:
>> 
>> IDENTIFIER: 
>>   'a' .. 'z' ( 'a' .. 'z' | '0' .. '9' | '_' | '$' | '#' | '.' | ' ' )* ; 
>> 

I guess I am really dense tonite but I do not see the problem here.

The 'F' in "FROM" is not a valid IDENTIFIER character.

So I suppose you get 6 tokens here, assuming Upper-Case letters
are recognized in some other lexer rule. (I guess, just like the
"SELECT" is not recognized as an IDENTIFIER).

the 6 tokens are (i think):
1) SELECT             // recognized by the Upper-Case rule
2) a field with name  // an IDENTIFIER
3) FROM               // recognized by the Upper-Case rule
4) a                  // an IDENTIFIER
5) T                  // recognized by the Upper-Case rule
6) able               // an IDENTIFIER

and we can make it be the 4 you want by:

IDENTIFIER: 
   'a' .. 'z' ( 'A' .. 'Z' | 'a' .. 'z' | '0' .. '9'
              | '_' | '$' | '#' | '.' | ' ' )* ; 

and I assume a lexer rule similar to:

KEYWORD options { testLiterals=true; } : 
   'A' .. 'Z' ( 'A' .. 'Z' | 'a' .. 'z' | '0' .. '9'
              | '_' | '$' | '#' | '.' | ' ' )* ;  // maybe not these

thus IDENTIFIERs begin with a lower case letter and KEYWORDs begin
with an upper case letter.

is that the way SQL works?

 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/