Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

memory allocation in collection of 1 million references in java

I created an ArrayList of 1 million MyItem objects and the memory consumed was 106mb(checked from Task Manager) But after adding the same list to two more list through addAll() method ,it takes 259mb. My question is I have added only the references to list , no new objects are created after that 1 million.why does the memory consumption increase eventhough LinkedList has been used (as it wont require contiguous memory blocks so no reallocation will be made)?

How to achieve this efficiently? Data passes through various lists in my program and consuming more than 1GB of memory.Similar scenario is presented above.

public class MyItem{
private String s;
private int id;
private String search;

public MyItem(String s, int id) {
    this.s = s;
    this.id = id;
}

public String getS() {
    return s;
}

public int getId() {
    return id;
}


public String getSearchParameter() {
    return search;
}

public void setSearchParameter(String s) {
    search = s;
}
}

public class Main{
    public static void main(String args[]) {
        List<MyItem> l = new ArrayList<>();
        List<MyItem> list = new LinkedList<>();
        List<MyItem> list1 = new LinkedList<>();

        for (int i = 0; i < 1000000 ; i++) {
            MyItem m = new MyItem("hello "+i ,i+1);
            m.setSearchParameter(m.getS());
            l.add(i,m);
        }

        list.addAll(l);

        list1.addAll(l);
        list1.addAll(list);

        Scanner s = new Scanner(System.in);
        s.next();//just not to terminate 
    }
}
like image 421
Jegan Babu Avatar asked Feb 07 '23 00:02

Jegan Babu


1 Answers

LinkedList is a doubly-linked list, so elements in the list are represented by nodes, and each node contains 3 references.

From Java 8:

private static class Node<E> {
    E item;
    Node<E> next;
    Node<E> prev;

    Node(Node<E> prev, E element, Node<E> next) {
        this.item = element;
        this.next = next;
        this.prev = prev;
    }
}

Since you use a lot of memory, you may not be using compressed OOP, so references might be 64-bit, i.e. 8 bytes each.

With an object header of 16 bytes + 8 bytes per reference, a node occupies 40 bytes. With 1 million elements, that would be 40 Mb.

Two lists is 80 Mb, and then remember that Java memory is segmented into pools and objects (the nodes) gets moved around, and your memory consumption of an additional 153 Mb now seems about right.

Note: An Arraylist would only use 8 bytes per element, not 40 bytes, and if you preallocate the backing array, which you can do since you know the size, you would save a lot of memory that way.

like image 188
Andreas Avatar answered Feb 09 '23 23:02

Andreas