Copy data between different databases (both are jdbc supported)

Question

I'd like to copy all data from my testing database (mysql) to a production database (hsqldb) I used hibernate and let it created tables in those database for me. So the structure/schema are the same.

Using mysql dump i can copy data between two mysql databases. but in my case the database are different, and i heard the generated sql from mysqldump wont work with hsqldb. Since both database type are support by jdbc/hibernate, are there any way/method/java library to copy data between them?

Answer 1

Sure, it is possible in a very easy way if the schemata are the same. And since you created both databases with the same Hibernate mapping they should be equal in the Entity sense.

You only need two Hibernate persistence units (datasources). If both are configured properly and you have the particular EntityManager instances handy, just go down to the Hibernate Session level - as far as I know JPA does not support that in this way (correct me if I'm wrong) - and replicate your source entity to your target database.

Because I like to work with Spring, I will use Spring Boot for the following example. Except of the configuration the replication step would be implemented the same with any Hibernate application.

I'm also using two PostgreSQL databases instead of an HSQLB only to keep it simple. Just extend the configuration part if your configurations drift apart, the only difference between my persistence units is the datasource url.

So first we need an entity to test the replication:

import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.Id;

@Entity
public class StorageEntry {

    @Id
    @GeneratedValue
    private Long id;

    private String someValue;

    // imagine getters and setter here

}

This is (the YAML version of) the configuration of the two datasources (see the second datasource url called targetDatabaseUrl), All other parts of the config will be used for both persistence units:

spring:
  datasource:
    url: jdbc:postgresql://localhost/postgres
    targetDatabaseUrl: jdbc:postgresql://localhost/postgres2
    username: <username>
    password: <password>
    driver-class-name: org.postgresql.Driver
  jpa:
    database-platform: org.hibernate.dialect.PostgreSQLDialect
    hibernate:
      ddl-auto: create-drop

The next part is the configuration class for the datasources:

import java.util.Properties;

import javax.persistence.EntityManager;
import javax.persistence.EntityManagerFactory;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.jdbc.datasource.DriverManagerDataSource;
import org.springframework.orm.jpa.JpaTransactionManager;
import org.springframework.orm.jpa.JpaVendorAdapter;
import org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean;
import org.springframework.transaction.PlatformTransactionManager;

@Configuration
public class PersistenceConfig {

    @Autowired
    private JpaVendorAdapter jpaVendorAdapter;

    @Value("${spring.datasource.url}")
    private String databaseUrl;

    @Value("${spring.datasource.targetDatabaseUrl}")
    private String targetDatabaseUrl;

    @Value("${spring.datasource.username}")
    private String username;

    @Value("${spring.datasource.password}")
    private String password;

    @Value("${spring.datasource.driver-class-name}")
    private String driverClassName;

    @Value("${spring.jpa.database-platform}")
    private String dialect;

    @Value("${spring.jpa.hibernate.ddl-auto}")
    private String ddlAuto;

    @Bean
    public EntityManager sourceEntityManager() {
        return sourceEntityManagerFactory().createEntityManager();
    }

    @Bean
    public EntityManager targetEntityManager() {
        return targetEntityManagerFactory().createEntityManager();
    }

    @Bean
    public EntityManagerFactory sourceEntityManagerFactory() {
        return createEntityManagerFactory("source", databaseUrl);
    }

    @Bean
    public EntityManagerFactory targetEntityManagerFactory() {
        return createEntityManagerFactory("target", targetDatabaseUrl);
    }

    @Bean
    public PlatformTransactionManager sourceTransactionManager() {
        return new JpaTransactionManager(sourceEntityManagerFactory());
    }

    @Bean
    public PlatformTransactionManager targetTransactionManager() {
        return new JpaTransactionManager(targetEntityManagerFactory());
    }

    private EntityManagerFactory createEntityManagerFactory(final String persistenceUnitName,
            final String databaseUrl) {
        final LocalContainerEntityManagerFactoryBean entityManagerFactory = new LocalContainerEntityManagerFactoryBean();

        final DriverManagerDataSource dataSource = new DriverManagerDataSource(databaseUrl, username, password);
        dataSource.setDriverClassName(driverClassName);
        entityManagerFactory.setDataSource(dataSource);

        entityManagerFactory.setJpaVendorAdapter(jpaVendorAdapter);
        entityManagerFactory.setPackagesToScan("com.example.model");
        entityManagerFactory.setPersistenceUnitName(persistenceUnitName);

        final Properties properties = new Properties();
        properties.setProperty("hibernate.dialect", dialect);
        properties.setProperty("hibernate.hbm2ddl.auto", ddlAuto);
        entityManagerFactory.setJpaProperties(properties);

        entityManagerFactory.afterPropertiesSet();
        return entityManagerFactory.getObject();
    }

}

Now you can use the different entity managers to simply read and write your data from one datasource to another. To show that here is a small test case:

import static org.hamcrest.CoreMatchers.notNullValue;
import static org.hamcrest.CoreMatchers.nullValue;
import static org.hamcrest.MatcherAssert.assertThat;

import javax.persistence.EntityManager;
import javax.persistence.PersistenceContext;

import org.hibernate.ReplicationMode;
import org.hibernate.Session;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;
import org.springframework.transaction.annotation.Transactional;

import com.example.model.StorageEntry;

@SpringBootTest
@RunWith(SpringRunner.class)
@Transactional(transactionManager = "targetTransactionManager")
public class ReplicationTests {

    @PersistenceContext(unitName = "source")
    private EntityManager sourceEntityManager;

    @PersistenceContext(unitName = "target")
    private EntityManager targetEntityManager;

    @Test
    public void copyEntityBetweenPersistenceUnits() {
        final StorageEntry entityToCopy = new StorageEntry();
        entityToCopy.setSomeValue("copyMe!");
        sourceEntityManager.persist(entityToCopy);

        final Long id = entityToCopy.getId();

        final StorageEntry sourceEntity = sourceEntityManager.find(StorageEntry.class, id);
        assertThat("Entity should exist in default schema!", sourceEntity, notNullValue());

        StorageEntry targetEntity = targetEntityManager.find(StorageEntry.class, id);
        assertThat("Target schema should not contain the entity, yet!", targetEntity, nullValue());

        final Session hibernateSession = targetEntityManager.unwrap(Session.class);
        hibernateSession.replicate(sourceEntity, ReplicationMode.OVERWRITE);

        targetEntityManager.flush();
        targetEntityManager.clear();

        targetEntity = targetEntityManager.find(StorageEntry.class, id);
        assertThat("Entity should be copied now!", targetEntity, notNullValue());
    }

}

Finally, choose one of the possible replication modes which fits your needs.

That's all. You can even use a transaction, just decide for one of both persistence units and leverage it's transaction manager like the test does with @Transactional(transactionManager = "targetTransactionManager").