[antlr-interest] Testing lexer grammars with gunit

[Gavin Lambert]

At 20:54 24/11/2009, Leon Su wrote:
>gUnit P;
>lexical-rule-name:
>"input" OK
>...
>
>By the way, the next release of gUnit will allow you to test a 
>lexer grammar individually with the syntax: gUnit lexer L;

Does gUnit only support that kind of limited testing?  (I ask out 
of ignorance; I've never really looked at it.)

For lexer rules in particular, "OK" is a fairly meaningless 
test.  What'd be better is something like:

gUnit P;
LEXER:
   "abc" ID
   "abc123" ID
   "123" INT
   "a+b" ID["a"] PLUS ID["b"]
   "a--b" ID DECREMENT ID
   "a- -b" ID MINUS MINUS ID

etc.  Then you could do lexer-only testing for lexer grammars and 
lexer-and-parser testing for combined grammars.