[antlr-interest] Inserting tokens?

[Greg Smolyn]

Hi everyone, hopefully this is a stupid question with a similarly easy  
answer.

I have a beautiful CSS grammar that I carefully built based on W3C  
specs (my first mistake!).  However, it seems those crazy web  
programmers are one step ahead and have done some crazy things. :)

So basically, I have some lexer rules defining class and id parameters:

HASH : '#' F_IDENT;
CLASS: '.' F_IDENT;

The grammar is one that uses spaces, so I have a rule

selector:  simple_selector (combinator simple_selector)*;

where combinator can be a space (which defines the standard combinator).

So, generally, you're supposed to do things like "table td #someId" or  
"table td .someClass".  In this case, they've gone and put "table  
td#someId", and similarly for class (where table and td are IDENT  
tokens).

Does it makes sense to modify the IDENT token so it looks like

IDENT: F_IDENT | F_IDENTWITHHASH | F_IDENTWITHCLASS ;

and in the latter two actually emit an IDENT, Space, and HASH or CLASS  
token?

Thanks for your help!

-greg