Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Writing .npy (numpy binary format) from java

Is there a library to create npy file in java?

I'm looking for a method to write large matrices in java, to be read using python code.

npy seems like a good option, as it doesn't add additional dependencies in the python side, and the format is documented.

I considered hdf5 format, but the dependency on native libraries makes the deployment harder.

like image 289
Ophir Yoktan Avatar asked Sep 10 '13 19:09

Ophir Yoktan


2 Answers

We've encountered the same problem a while ago and implemented both NPY and NPZ formats in Kotlin as part of the npy library.

The current version (0.2.0) has a number of limitations. Specifically, it doesn't support

  • multidimensional arrays
  • structured arrays
  • unsigned integral types and some others.

Java code:

Path filePath = new File("my.npz").toPath();
int[] values = {1, 2, 3, 4, 5, 6};
int[] shape = {2, 3};
try (NpzFile.Writer writer = NpzFile.write(filePath, true)) {
    writer.write("my-entry", values, shape);
}

Maven dependency:

<dependency>
    <groupId>org.jetbrains.bio</groupId>
    <artifactId>npy</artifactId>
    <version>0.3.3</version>
</dependency>
like image 95
Sergei Lebedev Avatar answered Oct 28 '22 16:10

Sergei Lebedev


I don't think there is a library to do this but you could look at the specification, or possibly use the matlab format as an intermediate, for which there appears to be at least one java library:

http://www.mathworks.com/matlabcentral/fileexchange/10759-jmatio-matlabs-mat-file-io-in-java

I've never done this, but it might work for you and then you can read those files via:

http://docs.scipy.org/doc/scipy/reference/generated/scipy.io.loadmat.html

like image 24
JoshAdel Avatar answered Oct 28 '22 15:10

JoshAdel