Write and read multiple byte[] in file with Java

Question

I have to write byte arrays in a file. I can't do it in one time so I can't put my arrays in a container. Also the size of my arrays is variable. Secondly, the file is very huge, so I have to split it, in order to read it array by array.

How can I do that ? I tried to write line by line my byte arrays but I haven't been able. How can I put a separator between my arrays and after split it over this separator ?

EDIT :

I tried this :

ByteArrayOutputStream bos = new ByteArrayOutputStream();
ObjectOutput out = new ObjectOutputStream(bos);
out.writeObject(byteArray);

But, I execute this code several times, so the ObjectOutputStream adds each time a new header which corrupt the file.

I also try :

out.write(byteArray);

but I couldn't separate my arrays. So I tried to append a '\n', which didn't work. After I was looking for library like FileUtils in order to write byte[] line by line but I didn't find.

Answer 1

You can use existing collections Like e.g. List to maintain List of byte[] and transfer it

    List<byte[]> list = new ArrayList<byte[]>();
    list.add("HI".getBytes());
    list.add("BYE".getBytes());

    ObjectOutputStream out = new ObjectOutputStream(new FileOutputStream(
            "test.txt"));
    out.writeObject(list);

    ObjectInputStream in = new ObjectInputStream(new FileInputStream(
            "test.txt"));
    List<byte[]> byteList = (List<byte[]>) in.readObject();

    //if you want to add to list you will need to add to byteList and write it again
    for (byte[] bytes : byteList) {
        System.out.println(new String(bytes));
    }

Output:

   HI
   BYE

Another option is use RandomAccessFile. Which will not force you to read complete file and you can skip the data that you don't want to read.

     DataOutputStream dataOutStream = new DataOutputStream(
            new FileOutputStream("test1"));
    int numberOfChunks = 2;
    dataOutStream.writeInt(numberOfChunks);// Write number of chunks first
    byte[] firstChunk = "HI".getBytes();
    dataOutStream.writeInt(firstChunk.length);//Write length of array a small custom protocol
    dataOutStream.write(firstChunk);//Write byte array

    byte[] secondChunk = "BYE".getBytes();
    dataOutStream.writeInt(secondChunk.length);//Write length of array
    dataOutStream.write(secondChunk);//Write byte array

    RandomAccessFile randomAccessFile = new RandomAccessFile("test1", "r");
    int chunksRead = randomAccessFile.readInt();
    for (int i = 0; i < chunksRead; i++) {
        int size = randomAccessFile.readInt();
        if (i == 1)// means we only want to read last chunk
        {
            byte[] bytes = new byte[size];
            randomAccessFile.read(bytes, 0, bytes.length);
            System.out.println(new String(bytes));
        }
        randomAccessFile.seek(4+(i+1)*size+4*(i+1));//From start so 4 int + i* size+ 4* i ie. size of i
    }

Output:

BYE