AbstractSpliterator implementation with unknown size throws OutOfMemoryError: Java heap space

Question

I have got a java.lang.OutOfMemoryError: Java heap space when I use an AbstractSpliterator implementation, which reports an unknown size.

In this case I defined a class StreamCollapse which extends AbstractSpliterator and merges series of adjacent elements in tryAdvance() implementation. Its constructor calls the super constructor as super(Long.MAX_VALUE, source.characteristics()).

Regarding the API documentation I was expecting that using a Long.MAX_VALUE indicates an unknown size. However, it seems it is trying to allocate memory with that size, instead.

Why is it trying to allocate that space? What value should I use for estimated size?

Here it is an example test:

Stream<Integer> nrs = Stream.of(3, 3, 5, 5, 3, 3, 3, 4, 4, 4 ,5 , 5);
Integer [] expected = {3, 5, 3, 4, 5};
Object[] actual = collapse(nrs).toArray();
assertEquals(actual, expected);

And the collapse() method implementation:

static <T> Stream<T> collapse(Stream<T> source) {
    return StreamSupport.stream(
            new StreamCollapse<T>(source.spliterator()), false);
}

class StreamCollapse<T> extends AbstractSpliterator<T> implements Consumer<T> {

    private final Spliterator<T> source;
    private T curr = null;

    StreamCollapse(Spliterator<T> source) {
        super(Long.MAX_VALUE, source.characteristics());
        this.source = source;
    }

    @Override
    public boolean tryAdvance(Consumer<? super T> action) {
        T prev = curr;
        boolean hasNext;
        while ((hasNext = source.tryAdvance(this)) && curr.equals(prev)) { }
        if(hasNext) action.accept(curr);
        return hasNext;
    }

    @Override
    public void accept(T item) {
        curr = item;
    }
}

Answer 1

You should remove the characteristics from your composited spliterator, for example:

// an unknown spliterator shouldn't having SIZED | SUBSIZED  characteristics
//                                             v  
super(Long.MAX_VALUE, source.characteristics() & (~(SIZED | SUBSIZED)));

WHEN a Spliteartor is a SIZED Spliteartor, Spliterator#getExactSizeIfKnown will be used by stream to create an array.

Characteristic value signifying that the value returned from estimateSize() prior to traversal or splitting represents a finite size that, in the absence of structural source modification, represents an exact count of the number of elements that would be encountered by a complete traversal.

IF the stream run in parallel the Stream#toArray will throws a IllegalArgumentException if the estimateSize >= Long.MAX_VALUE - 8.

IF the stream is a sequentially stream the Stream#toArray will grows its internal array capacity to the estimateSize.