Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Connection pool for dynamic database connections

The problem setup is based on a webservice (Spring/Java, Tomcat7 and MySql) where every user gets their own database, hence each request needs their own connection. As all databases are created dynamically during runtime, configuring them statically before startup is not an option.

To optimise database connection usage, an implementation of a database connection pool would be great, right?

With Java/Spring: How would I create a connection pool for dynamic databases? I am a bit struck by the lack of clean options here!

Problem: Tomcat's Connection Pool (and as far as i understand C3P0 as well) treats each new DataSource instance as a whole new connection pool -> stack-reference

  1. Is it a good idea to create a static datasource with a generic MySql connection (without specifing the database on connection) and use a connection pool with this datasource together with adapted SQL statements?
    stack-reference
  2. What about developing a custom persistent database based datasource pool? Any experience with performance here? Any advice? Any libraries that do that?
  3. Or would it be feasable to workaround Tomcat's DataSource problem by creating Tomcat JNDI Datasources dynamically by manipulating it's context.xml dynamically from Java?
  4. I can't believe that there aren't more plain/simple solutions for this. Grails/Hibernate struggles with this, Java/JDBC struggles with this, ... is it such a rare use-case to separate userdata on a user basis by creating user specific databases dynamically? If so, what would be a better setup?

EDIT

  1. Another option is the suggestion from @M.Deinum to use a single configured datasource and dynamically hotswap it for the right connection ->M.Deinum Blog and stack-reference.
    How does that perform with a connection pool like the ones above?
like image 357
N.R. Avatar asked May 27 '15 09:05

N.R.


People also ask

What is a connection pool in database?

Database connection pooling is a way to reduce the cost of opening and closing connections by maintaining a “pool” of open connections that can be passed from database operation to database operation as needed.

What is the ideal DB connection pool size?

For optimal performance, use a pool with eight to 16 connections per node. For example, if you have four nodes configured, then the steady-pool size must be set to 32 and the maximum pool size must be 64. Adjust the Idle Timeout and Pool Resize Quantity values based on monitoring statistics.

What is database connection pooling advantages of using a connection pool?

Using connection pools helps to both alleviate connection management overhead and decrease development tasks for data access. Each time an application attempts to access a backend store (such as a database), it requires resources to create, maintain, and release a connection to that datastore.

What is connection pool in SQL?

A connection pool is created for each unique connection string. When a pool is created, multiple connection objects are created and added to the pool so that the minimum pool size requirement is satisfied. Connections are added to the pool as needed, up to the maximum pool size specified (100 is the default).


2 Answers

I believe that HikariCP works without having to specify a single database.

like image 141
ErikHH Avatar answered Sep 25 '22 11:09

ErikHH


Once the databases are created in runtime, you have to create the pools also in runtime. I am afraid the spring infrastructure is not giving you any help here, as it is tuned for the usual static use case.

I'd have a map of pools:

  • have a
     Map < connectionUrlString,List< c3poPool > > map
  • when requesting a connection, get the corresponding c3po pool from the map
  • and you can get the best of both worlds, since the real connection pool for each dynamically created database is handled by a c3po instance, but you can create new instances in runtime

This works as a low-level solution. If you want to go further, you can wrap this logic into a db connection provider, and register that as a "driver". This way any part of your application requests a new connection, you can just return one connection from the existing pools (and if a totally new connection is requested, create a new pool for that).

like image 37
Gee Bee Avatar answered Sep 21 '22 11:09

Gee Bee