[antlr-interest] Any plans of next ANTLR Release

[Graham Wideman]

Hi Ter,


>not soon. i'm done with analysis and working on new code gen architecture. new lexer stuff is amazing.  my goal will be super small code for parser/lexers.  I'll have some results to show soon on lexer.  E.g., current v3 lexer generated from Java.g is 136k bytes of java code.  I'm hoping to get that down to a few k in bytecodes for an NFA VM that's close in speed to an optimized DFA.  The prototype NFA VM itself (needs more opcodes though) is only about 600 bytes of Java bytecodes. :)


This prompts me to wonder how debuggable these lexers will be?  Currently a certain amount of troubleshooting of lexing/parsing can be done by inspecting the generated lexer source, single-stepping it and so on.

If you move to encoding the lexer logic in bytecodes, does the generated lexer source become an inscrutable black box?  Or is there still meaningful source code to examine, trace etc?

-- Graham