ClassFormatError in java 8?

Question

I was making a class similar to java.util.LinkedList, and got a completely unexpected ClassFormatError. My IDE shows no warnings. FYI, I am using Java 8u20. Update: fixed in Java 8u60.

T̶h̶i̶s̶ ̶i̶n̶c̶l̶u̶d̶e̶s̶ ̶a̶l̶l̶ ̶r̶e̶l̶e̶v̶a̶n̶t̶ ̶m̶e̶t̶h̶o̶d̶s̶:̶ Updated example as fully compilable:

import java.io.Serializable;
import java.util.*;
import java.util.function.Function;

public class Foo<E> implements Deque<E>, Serializable {
    private static final long serialVersionUID = 0L;

    private final Node sentinel = sentinelInit();
    private final Iterable<Node> nodes = (Iterable<Node> & Serializable) () -> new Iterator<Node>() {
        @SuppressWarnings("UnusedDeclaration")
        private static final long serialVersionUID = 0L;

        private Node next = sentinel.next;

        @Override
        public boolean hasNext() {
            return next != sentinel;
        }

        @Override
        public Node next() {
            if (!hasNext()) {
                throw new NoSuchElementException();
            }
            Node old = next;
            next = next.next;
            return old;
        }

        @Override
        public void remove() {
             if (next.previous == sentinel) {
                throw new IllegalStateException();
            }
            removeNode(next.previous);
        }
    };

    @Override
    public boolean isEmpty() {
        return false;
    }

    @Override
    public Object[] toArray() {
        return new Object[0];
    }

    @Override
    public <T> T[] toArray(T[] a) {
        return null;
    }

    @Override
    public boolean containsAll(Collection<?> c) {
        return false;
    }

    @Override
    public boolean removeAll(Collection<?> c) {
        return false;
    }

    @Override
    public boolean retainAll(Collection<?> c) {
        return false;
    }

    @Override
    public void clear() {

    }

    @Override
    public void addFirst(E e) {

    }

    @Override
    public void addLast(E e) {

    }

    @Override
    public boolean offerLast(E e) {
        return false;
    }

    @Override
    public E removeFirst() {
        return null;
    }

    @Override
    public E removeLast() {
        return null;
    }

    @Override
    public E pollFirst() {
        return null;
    }

    @Override
    public E getFirst() {
        return null;
    }

    @Override
    public E getLast() {
        return null;
    }

    @Override
    public E peekFirst() {
        return null;
    }

    @Override
    public boolean removeFirstOccurrence(Object o) {
        return false;
    }

    @Override
    public boolean removeLastOccurrence(Object o) {
        return false;
    }

    @Override
    public E remove() {
        return null;
    }

    @Override
    public E element() {
       return null;
    }

    @Override
    public void push(E e) {

    }

    @Override
    public E pop() {
        return null;
    }

    @Override
    public boolean contains(Object o) {
        return false;
    }

    @Override
    public boolean offerFirst(E e) {
        return false;
    }

    @Override
    public E pollLast() {
        return null;
    }

    @Override
    public E peekLast() {
        return null;
    }

    @Override
    public boolean offer(E e) {
        Node node = new Node(e);
        sentinel.previous.next = node;
        node.previous = sentinel.previous;
        sentinel.previous = node;
        node.next = sentinel;
        return true;
    }

    @Override
    public E poll() {
        return null;
    }

    @Override
    public E peek() {
        return null;
    }

    @Override
    public boolean remove(Object o) {
        for (Node node : nodes) {
            if (node.value.equals(o)) {
                removeNode(node);
                return true;
            }
        }
        return false;
    }

    @Override
    public int size() {
        return 0;
    }

    @Override
    public Iterator<E> descendingIterator() {
        return null;
    }

    @Override
    public Iterator<E> iterator() {
        return new Iterator<E>() {
            private final Iterator<Node> backingIter = nodes.iterator();

            @Override
            public boolean hasNext() {
                return backingIter.hasNext();
            }

            @Override
            public E next() {
                return backingIter.next().value;
            }

            @Override
            public void remove() {
                backingIter.remove();
            }
        };
    }

    private Node sentinelInit() {
        Node sentinel = new Node();
        sentinel.next = sentinel;
        sentinel.previous = sentinel;
        return sentinel;
    }

    private void removeNode(Node node) {
        node.previous.next = node.next;
        node.next.previous = node.previous;
    }

    private class Node implements Serializable {
        private static final long serialVersionUID = 0L;
        public E value;
        public Node next;
        public Node previous;

        public Node(E value) {
            this.value = value;
        }

        public Node() {
            this(null);
        }
    }

    public static <I, O> List<O> map(Function<? super I, O> function, Iterable<I> objects) {
        ArrayList<O> returned = new ArrayList<>();
        for (I obj : objects) {
            returned.add(function.apply(obj));
        }
        return returned;
    }

    @Override
    public boolean addAll(Collection<? extends E> c) {
        boolean ret = false;
        for (boolean changed : map(this::add, c)) {
            if (changed) {
                ret = true;
            }
        }
        return ret;
    }

    @Override
    public boolean add(E e) {
        if (!offer(e)) {
            throw new IllegalStateException();
        }
        return true;
    }

    public static void main(String[] args) {
        Foo<String> list = new Foo<>();
        System.out.println("Constructed list");
        list.addAll(Arrays.asList("a", "B", "c"));
        System.out.println("Added a, B and c.");
        list.forEach(System.out::println);
        list.remove("B");
        list.forEach(System.out::println);
    }
}

Here is the output:

Constructed list
Added a, B and c.
Exception in thread "main" java.lang.ClassFormatError: Duplicate field name&signature in class file uk/org/me/Foo$1
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:760)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:455)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:367)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at uk.org.me.Foo.lambda$new$c83cc381$1(Foo.java:18)
    at uk.org.me.Foo$$Lambda$1/1392838282.iterator(Unknown Source)
    at uk.org.me.Foo$2.<init>(Foo.java:222)
    at uk.org.me.Foo.iterator(Foo.java:221)
    at java.lang.Iterable.forEach(Iterable.java:74)
    at uk.org.me.Foo.main(Foo.java:300)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:483)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)

Answer 1

Your code is valid, and the ClassFormatError is due to a bug in the Java 8 compiler. I think you have uncovered an edge case: loading the class file generated for a lambda expression body which contains a variable with the same name as a method that is being overriden within that body may fail.

I pasted your code into the latest versions of Intellij Idea (13.1.5), NetBeans (8.0.1) and Eclipse (Kepler SR2). The code compiled in all cases. When run, it failed with that ClassFormatError in NetBeans and Idea, yet it ran fine in Eclipse.

The stack trace shows the Error is arising from a failed attempt to load the class generated by the lamba expression assigned to instance variable Foo.nodes. The root cause is that within the lambda body assigned to Foo.nodes you have a variable next with the same name as the overriden method next().

As you have already noted, if you rename the variable in the declaration private Node next = sentinel.next; to some unique value (e.g. next2) the problem goes away. Similarly, renaming it to a different overriden method (i.e. remove or hasNext) causes the problem to reappear. The message accompanying the ClassFormatError ("Duplicate field name&signature") exactly describes the problem, but does not explain why this should be such a precisely defined runtime error at all; the code compiled, and having the same name for a method and a variable is legal (if unwise). Prefixing next with "this." doesn't solve the problem.

Running javap on the failing Idea class file (foo/Foo$1 in my case) shows this:

C:\Idea\Java8\Foo\out\production\Foo\foo>"C:\Program Files\Java\jdk1.8.0_20\bin\javap.exe" Foo$1.class
Compiled from "Foo.java"
class foo.Foo$1 implements java.util.Iterator<foo.Foo<E>.Node> {
  final foo.Foo this$0;
  foo.Foo$1(foo.Foo);
  public boolean hasNext();
  public foo.Foo<E>.Node next();
  public void remove();
} 

However, running javap on the corresponding Eclipse class which ran fine shows an extra entry:

public java.lang.Object next();

That extra method entry also magically appears in the Idea class file if the variable next within the lambda body is renamed to something unique, allowing the code to run. I suggest you report this as a compiler bug to JetBrains.

Also, as an unrelated issue, the Iterator interface implements remove() as a default method in Java 8. Since you never call Iterator.remove() you can delete your two implementations.

UPDATE: I raised a bug (JDK-8080842) for this issue with Oracle, and there will be a fix for it in release 1.8.0_60.

UPDATE TO THE UPDATE (8/27/15): Oracle have fixed this issue in all releases from 8u60 onwards. I verified the fix against 8u60. Bug JDK-8080842 refers.

Answer 2

Both variables inside the nodes iterable and the private class Node called next are of the same type (Node) and have the same name: inside that private class, they are both in scope, but cannot be disambiguated.

So, the issue is not the next field and next() method, but the next named variables both in Iterable<Node> nodes and your private class Node.

If you refactor sentinel.next to some other name:

private final Iterable<Node> nodes = (Iterable<Node> & Serializable) () -> new Iterator<Node>() {
    @SuppressWarnings("UnusedDeclaration")
    private static final long serialVersionUID = 0L;

    private Node snext = sentinel.next;  // refactor `next` to `snext`
    ...

It will compile and run.

As a personal note, I would discourage you writing code such as this: it's incredibly difficult to read and figure out what it does.

Hope this helps!