I currently have a very large file which contains a few million lines of entries, and want them inserted into a database. The connection established from java to SQL works as I have tried inserting the data singularly and it works, however, when I switched to using executeBatch and addBatch, it seems to loop though but not populating anything into my database.
Code is as follows:
import java.io.BufferedReader;
import java.io.FileReader;
import java.sql.Connection;
import java.sql.DatabaseMetaData;
import java.sql.DriverManager;
import java.sql.SQLException;
public class fedOrganiser6 {
private static String directory = "C:\\Users\\x\\Desktop\\Files\\";
private static String file = "combined.fed";
private static String mapperValue = "";
public static void main(String[] args) throws Exception {
Connection conn = null;
try {
BufferedReader mapper = new BufferedReader(new FileReader(directory + file));
String dbURL = "jdbc:sqlserver://localhost\\SQLExpress;database=TIMESTAMP_ORGANISER;integratedSecurity=true";
String user = "sa";
String pass = "password";
conn = DriverManager.getConnection(dbURL, user, pass);
if (conn != null) {
DatabaseMetaData dm = (DatabaseMetaData) conn.getMetaData();
System.out.println("Driver name: " + dm.getDriverName());
System.out.println("Driver version: " + dm.getDriverVersion());
System.out.println("Product name: " + dm.getDatabaseProductName());
System.out.println("Product version: " + dm.getDatabaseProductVersion());
System.out.println("clearing database");
conn.createStatement().executeUpdate("truncate table TimestampsStorage");
System.out.println("bulk insert into database");
System.out.println("complete");
int i = 0;
int records = 0;
String query = "INSERT INTO TimestampsStorage " + "values(" + "'" + mapperValue.toString() + "'"+ ")";
conn.prepareStatement(query);
for (mapperValue = mapper.readLine(); mapperValue != null; mapperValue = mapper.readLine()) {
i++;
records++;
System.out.println("Batching " + records + " records...");
conn.createStatement().addBatch(query);
if (i == 100000) {
conn.createStatement().executeBatch();
i = 0;
}
}
}
conn.createStatement().executeBatch();
conn.createStatement().close();
System.out.print("Done");
} catch (SQLException ex) {
ex.printStackTrace();
} finally {
try {
if (conn != null && !conn.isClosed()) {
conn.close();
}
} catch (SQLException ex) {
ex.printStackTrace();
}
}
}
}
createStatement() creates a new statement object, so you're execute a different statement than the one you're batching on. You should create the PreparedStatement once, add several batches to it, and then execute on the same object:
String query = "INSERT INTO TimestampsStorage VALUES (?)";
PreparedStatement ps = conn.prepareStatement(query);
for (mapperValue = mapper.readLine();
mapperValue != null;
mapperValue = mapper.readLine()) {
i++;
records++;
System.out.println("Batching " + records + " records...");
ps.setString(1, mapperValue);
ps.addBatch();
if (i == 100000) {
ps.executeBatch();
i = 0;
}
}
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With