[antlr-interest] StringTemplate test case: Capitalizing all words in text

Егор Абрамович egor.abramovich at googlemail.com
Thu Sep 11 09:12:12 PDT 2008


Hello,

In my test I am trying to create a grammar that should capitalize all
words in text.
Supposing text contains of english characters, punctuation marks and numbers.
Here is my grammar (with line numbers added):

-------------------------------->8--- start of UpperAll.g

01: grammar UpperAll;
02: options {output=template; rewrite=true;}
03:
04: sent_en  :   ( en+=en_word PUNKT? INT? )+ //->
template(words={$en}) "<words; separator=\" \">"
05:          ;
06:
07: en_word   :   ('A'..'Z' | 'a'..'z' )+ -> template(word={
TestUpperAll.caps($en_word.text) }) "<word>"
08:           ;
09:
10: PUNKT     :   '.' | ',' | '!' | '?' | '(' | ')';
11: INT       :   '0'..'9'+ ;
12: NEWLINE  :   '\r'? '\n' ;
13: WS       :   (' '|'\t'|'\n'|'\r')+ {$channel = HIDDEN;} ;

-------------------------------->8--- end of UpperAll.g

When trying to generate, I am getting an error message:

Check Grammar reported some errors:
error(201):UpperAll.g:4:40: The following alternative can never be matched: 2

Here is console output:

[10:40:53] warning(200): UpperAll.g:4:28: Decision can match input
such as "PUNKT" using multiple alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
[10:40:53] warning(200): UpperAll.g:4:35: Decision can match input
such as "INT" using multiple alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
[10:40:53] warning(200): UpperAll.g:4:40: Decision can match input
such as "{EOF, PUNKT..INT}" using multiple alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
[10:40:53] error(201): UpperAll.g:4:40: The following alternatives can
never be matched: 2

[10:40:53] warning(200): UpperAll.g:7:36: Decision can match input
such as "PUNKT" using multiple alternatives: 1, 2, 3
As a result, alternative(s) 3,2 were disabled for that input
[10:40:53] warning(200): UpperAll.g:7:36: Decision can match input
such as "INT" using multiple alternatives: 1, 2, 3
As a result, alternative(s) 3,2 were disabled for that input
[10:40:53] warning(200): UpperAll.g:7:36: Decision can match input
such as "EOF" using multiple alternatives: 1, 2, 3
As a result, alternative(s) 3,2 were disabled for that input
[10:40:53] error(201): UpperAll.g:7:36: The following alternatives can
never be matched: 2,3


When commenting out template generation in line 7, like below, grammar
is generating code successfully, but generated UpperAllParser.java
contains compilation errors:

07: en_word   :   ('A'..'Z' | 'a'..'z' )+ // -> template(word={
TestUpperAll.caps($en_word.text) }) "<word>"

I guess that problem is somewhere in line 4 of grammar, but I don't
know how to state this in a different way.

Here is a test rig for that grammar:

-------------------------------->8--- start of TestUpperAll.java

import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.PrintWriter;

import org.antlr.runtime.ANTLRInputStream;
import org.antlr.runtime.TokenRewriteStream;

public class TestUpperAll {

    public static void main(String[] args) throws Exception {
        int n = 1;
        FileInputStream in = new FileInputStream("test_upper_"+n+".txt");
        ANTLRInputStream input = new ANTLRInputStream(in);
        UpperAllLexer lexer = new UpperAllLexer(input);
        in.close();
        TokenRewriteStream tokens = new TokenRewriteStream(lexer);
        UpperAllParser parser = new UpperAllParser(tokens);
        parser.sent_en();
        saveStr("test_upper_"+n+"_out.txt", tokens.toString()); //
emit rewritten source
        System.out.println("Test "+n+" done");
    }

    /** Capitalize the first letter of string */
    public static String caps(String s) {
        System.out.println("caps: "+s);
        if (s == null || s.length() == 0)
            return s;
        StringBuffer b = new StringBuffer(s);
        b.setCharAt(0, Character.toUpperCase(b.charAt(0)));
        return b.toString();
    }

    /** Save string to file. */
    public static void saveStr(String fname, String text) throws IOException {
        PrintWriter out = new PrintWriter(new FileOutputStream(fname));
        out.print(text);
        out.close();
    }

}

-------------------------------->8--- end of TestUpperAll.java

I am using antlrworks-1.2.

Could you please advise how it should be done properly.

Thank you.

Best regards,
Egor Abramovich


More information about the antlr-interest mailing list