[antlr-interest] Using the TokenRewriteStream
Terence Parr
parrt at cs.usfca.edu
Wed Aug 1 10:40:21 PDT 2007
On Jul 30, 2007, at 1:53 PM, Susan Jolly wrote:
> However, I'm probably going to need modify
> TokenRewriteStream.toString() to
> use and return a buffer that is a List (of imaginary and real
> Tokens) rather
> than a StringBuffer.
Yeah, I have some thoughts on token stream rewrite on the blog.
> One of the things I need to be able to do is to interline the input
> (which
> is print text) with its braille translation. ("Interline" means
> alternating
> a line of print with a line of braille.) This requires adding line
> breaks to
> the input so as to keep it in sync with the braille since each line
> of the
> braille translation has a maximum length and must be terminated by
> a hard
> return. (The print lines typically have more characters than the
> braille
> lines.) This alone I could do by estimating the length of a braille
> line by
> totaling the String length of the real Tokens and then invoking
> toString
> again with a smaller "end" index if the imaginary Tokens added by the
> Rewrite make the line too long. (I can't put the line breaks in at
> a later
> point because I'd lose the synchronization with the input.)
>
> However, there are multiple ways to represent braille and a sighted
> user
> might want more than one of them. In other words, the general case
> is that
> the output consists of sets of lines where each set has one or more
> of the
> following: a line of print, a line of standard braille (that
> controls the
> line length), and possibly additional lines with other alternatives.
>
> This is (I think) easier to do if I associate each Token with an
> object that
> has all these multiple alternatives which is why I want to get a
> List of the
> Tokens rather than the concatenated text.
>
> The pictures at the tops of these two pages show what the output
> looks like
> except that the first one doesn't include the original print text.
> http://www.dotlessbraille.org/screencap.htm
> http://www.dotlessbraille.org/gemini.htm
Interesting. Well, a first pass to rewrite the token stream is a
decent idea. Then process the altered stream with another parser.
Ter
More information about the antlr-interest
mailing list