JAVA: TokenStreamRewriter#reduceToSingleOperationPerIndex consumes lots of memory in case of lots of RewriteOperations

[snuyanzin]

In Apache Flink (https://flink.apache.org/) we use antlr4 to split and rewrite generated code in order to make it fit 64Kb limitation from janino (https://janino-compiler.github.io/janino/)

The generated code might be huge enough and depending on rewrite rule there might be lots of RewriteOperations.

It looks like the more rewrite operations we have the way slower it works.

After looking into it seems the main issue is TokenStreamRewriter#getKindOfOps

antlr4/runtime/Java/src/org/antlr/v4/runtime/TokenStreamRewriter.java

Lines 586 to 595 in bd8f993

	protected <T extends RewriteOperation> List<? extends T> getKindOfOps(List<? extends RewriteOperation> rewrites, Class<T> kind, int before) {
	List<T> ops = new ArrayList<T>();
	for (int i=0; i<before && i<rewrites.size(); i++) {
	RewriteOperation op = rewrites.get(i);
	if ( op==null ) continue; // ignore deleted
	if ( kind.isInstance(op) ) {
	ops.add(kind.cast(op));
	}
	}
	return ops;

which tries to iterate multiple times through rewrites and to create a new collection.

Since we already have all the data in rewrite may be don't need to create new collections then