[antlr-interest] in v3.3, does the token stream mechanism slurp in the whole file?

Martin d'Anjou point14 at magma.ca
Thu Mar 24 17:18:50 PDT 2011


Hello,

According to:
http://v2kparse.blogspot.com/2008/06/first-pass-uploaded-to-sourceforce.html

the ANTLR v3.0 token stream mechanism tries to slurp in the whole source 
file, which is problematic with gigantic source files.

1) Does ANTLR v3.3 still try to read all the input at once?

2) Is this what BufferedTokenStream fixes?

Thanks,
Martin


More information about the antlr-interest mailing list