Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How can I quickly load a large txt file into BigInteger?

I'm importing a large text file, 17 million digits long and I'm using this code:

BufferedReader reader = new BufferedReader(new FileReader("test2.txt"));
String line = reader.readLine();
System.out.println("Done");
BigInteger num = new BigInteger(line);
System.out.println("Done Again");

It loads the file pretty much instantly and prints out 'Done' but it takes a long time (about an hour) for the String to be converted into a BigInteger, is there anything I can do to speed this up and quickly load the number?

like image 806
Samantha Clark Avatar asked May 16 '17 23:05

Samantha Clark


2 Answers

As an optimization, since BigInteger is Serializable, you could save it to a binary file once and speed up your loading considerably.

Loading a serialized object should be way faster than parsing a huge string everytime.

Use ObjectOutputStream to save your big integer and ObjectInputStream to read it back in.

like image 116
Paulo Mattos Avatar answered Sep 18 '22 01:09

Paulo Mattos


It is slow because new BigInteger(String) is doing radix conversion from decimal to binary, which is O(N2). Nothing you can do about that.

You could save either the object itself, via Serialization, or the byte array it is stored in, via BigInteger.toByteArray(). Either will load essentially instanteously.

like image 29
user207421 Avatar answered Sep 18 '22 01:09

user207421