How to define a data source programatically?

Question

I am trying to collect data source metrics using OpenTelemetry. According to the OpenTelemetry documentation, a DataSource instance has to be wrapped into an OpenTelemetryDataSource data source like this:

new OpenTelemetryDataSource(dataSource);

I went through the Quarkus documentation and haven't found a way to override data sources programmatically.

How to override data source programmatically in Quarkus? Or generally how can I connect OpenTelemetry to a data source in Quarkus?

UPDATE: When I set up JDBC URL with configuration properties (quarkus.datasource.jdbc.url=jdbc:otel:postgresql://localhost:5432/my_db) the application fails with NPE on start:

ERROR: Failed to start application (with profile dev)
java.lang.NullPointerException
        at io.quarkus.opentelemetry.runtime.QuarkusContextStorage.getVertxContext(QuarkusContextStorage.java:62)
        at io.quarkus.opentelemetry.runtime.QuarkusContextStorage.current(QuarkusContextStorage.java:54)
        at io.opentelemetry.context.Context.current(Context.java:86)
        at io.opentelemetry.instrumentation.jdbc.internal.OpenTelemetryStatement.wrapCall(OpenTelemetryStatement.java:277)
        at io.opentelemetry.instrumentation.jdbc.internal.OpenTelemetryPreparedStatement.executeQuery(OpenTelemetryPreparedStatement.java:53)
        at io.agroal.pool.wrapper.PreparedStatementWrapper.executeQuery(PreparedStatementWrapper.java:78)
        at org.flywaydb.core.internal.database.base.BaseDatabaseType.getSelectVersionOutput(BaseDatabaseType.java:195)
        at org.flywaydb.core.internal.database.cockroachdb.CockroachDBDatabaseType.handlesDatabaseProductNameAndVersion(CockroachDBDatabaseType.java:81)
        at org.flywaydb.core.internal.database.DatabaseTypeRegister.getDatabaseTypeForConnection(DatabaseTypeRegister.java:136)
        at org.flywaydb.core.internal.jdbc.JdbcConnectionFactory.<init>(JdbcConnectionFactory.java:69)
        at org.flywaydb.core.Flyway.execute(Flyway.java:510)
        at org.flywaydb.core.Flyway.migrate(Flyway.java:170)
        at io.quarkus.flyway.runtime.FlywayRecorder.doStartActions(FlywayRecorder.java:75)
        at io.quarkus.deployment.steps.FlywayProcessor$createBeansAndStartActions-1520831253.deploy_0(FlywayProcessor$createBeansAndStartActions-1520831253.zig:84)
        at io.quarkus.deployment.steps.FlywayProcessor$createBeansAndStartActions-1520831253.deploy(FlywayProcessor$createBeansAndStartActions-1520831253.zig:40)
        at io.quarkus.runner.ApplicationImpl.doStart(ApplicationImpl.zig:754)
        at io.quarkus.runtime.Application.start(Application.java:101)
        at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:101)
        at io.quarkus.runtime.Quarkus.run(Quarkus.java:66)
        at io.quarkus.runtime.Quarkus.run(Quarkus.java:42)
        at io.quarkus.runtime.Quarkus.run(Quarkus.java:119)
        at io.quarkus.runner.GeneratedMain.main(GeneratedMain.zig:29)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at io.quarkus.runner.bootstrap.StartupActionImpl$1.run(StartupActionImpl.java:98)
        at java.base/java.lang.Thread.run(Thread.java:829)

Answer 1

[EDIT]

Never mind, it looks like the quarkus framework actually does offer a way to swap data source implementation. It is explicitly mentioned in the opentelemetry documentation. https://github.com/open-telemetry/opentelemetry-java-instrumentation/tree/main/instrumentation/jdbc/library

Consult the quarkus framework documentation and find out how to configure it to allow manual DataSource instantiation. In the DataSource provider class that you will need to create simply read the properties file yourself (the same one that the quarkus framework reads) that contains all the database connection details, get the appropriate keys and instantiate AgroalDataSource objects as shown below in this answer, wrap them in OpenTelemetryDataSource and return the OpenTelemetryDataSource object.

Unfortunately data sources are NOT database connections, you simply cannot force your quarkus framework to use your programmatically defined data sources by manipulating jdbc url in properties file.

As per the documentation found at https://quarkus.io/guides/datasource quarkus uses agroal datasources https://github.com/agroal/agroal

Consult agroal documentation to find out how to construct datasources https://github.com/agroal/agroal/tree/master/agroal-api Here's what I found:

@SuppressWarnings( "ALL" )
class AgroalSampleUsage {

    public static void main() {
        AgroalDataSourceConfigurationSupplier configuration = new AgroalDataSourceConfigurationSupplier()
                .dataSourceImplementation( DataSourceImplementation.AGROAL )
                .metricsEnabled( false )
                .connectionPoolConfiguration( cp -> cp
                        .minSize( 5 )
                        .maxSize( 20 )
                        .initialSize( 10 )
                        .connectionValidator( defaultValidator() )
                        .acquisitionTimeout( ofSeconds( 5 ) )
                        .leakTimeout( ofSeconds( 5 ) )
                        .validationTimeout( ofSeconds( 50 ) )
                        .reapTimeout( ofSeconds( 500 ) )
                        .connectionFactoryConfiguration( cf -> cf
                                .jdbcUrl( "jdbc:h2:mem:test" )
                                .connectionProviderClassName( "org.h2.Driver" )
                                .autoCommit( false )
                                .jdbcTransactionIsolation( SERIALIZABLE )
                                .principal( new NamePrincipal( "username" ) )
                                .credential( new SimplePassword( "secret" ) )
                        )
                );

        try ( AgroalDataSource dataSource = AgroalDataSource.from( configuration ) ) {
            Connection connection = dataSource.getConnection();
            connection.close();
        } catch ( SQLException e ) {
            System.out.println( "Oops! " + e.getMessage() );
        }
    }
}

Unless the framework in question is configurable enough to allow you to swap data source implementation, i.e. not use argoal, but use another one. What you are trying to achieve is not possible without modifying the framework

If however you are looking to access the datasoruces instantiated by the framework

As per the documentation found at https://quarkus.io/guides/datasource#named-datasource-injection

quarkus datasources can be retrieved via dependency injection.

public class MyDatasourceWrapper {

    private Object lock = new Object();

    @Inject
    @DataSource("users") // or ignore
    private AgroalDataSource usersDataSource;

    private OpenTelemetryDataSource wrappedDataSource;

    public AgroalDataSource getUsersDataSource() {
        return usersDataSource;
    }

    public OpenTelemetryDataSource getUsersOpenTelemetryDataSource() {
        if(wrappedDataSource == null) {
            synchronized (lock) {
                wrappedDataSource = new OpenTelemetryDataSource(usersDataSource);
            }
        }
        return wrappedDataSource;
    }
}

To be used like this :

public class ClassUsingDataSource {

    @Inject
    private MyDatasourceWrapper dataSourceWrapper;

    public void run() {
        OpenTelemetryDataSource dataSource = dataSourceWrapper.getUsersOpenTelemetryDataSource();
         // dataSource. use ()
    }
}

However the wrapped OpenTelemetryDataSource will not be used by the framework.

One thing you could do, is make the assumption that the parts of the framework that use the datasources instantiated by the framework itself will be used trough dependency injection and DataSoruce interface, in which case you can create an DataSource interceptor and wrap incoming DataSource objects to OpenTelemetryDataSource objects. but this aproach makes too may assumptions.

If you have access and are allowed to modify the quarkus framework I suggest creating a property in the properties file, call it data source wrapper, it's value should be a class that contains a method

public DataSource wrap(DataSource dataSource);

If you have this kind of access, also make sure that everywhere in the soruce code of the framework the interface DataSource is used instead of the class AgroalDataSource as this might cause ClassCastException

Answer 2

It seems that some workarounds have been proposed in this open issue, but programmatically configure the Quarkus datasource is still a feature request.

This other issue, although related to OpenTracing and not OpenTelemetry, seems to confirm that.

Probably the best approach will be trying to configure this integration using configuration properties.

According to the Quarkus documentation and following the example of OpenTracing, please, try the following configuration:

quarkus.datasource.db-kind=postgresql
quarkus.datasource.jdbc.driver=io.opentelemetry.instrumentation.jdbc.OpenTelemetryDriver
quarkus.datasource.jdbc.url=jdbc:otel:postgresql://localhost:5432/hibernate_orm_test
# other configuration properties ...

Please, note the prefix in the URL. The example is for PostgreSql, but it should be very similar for other databases.

The configuration is inspired by the OpenTelemetry documentation.

According to your comments, this configuration conflicts with the Flyway database migration process carried out by Quarkus on initialization.

In order to solve the problem you can disable the Quarkus Flyway migrations at startup and perform the migration programatically.

As described in the Quarkus documentation, you can achieve this behavior by first setting the configuration property quarkus.flyway.migrate-at-start to false (it is true by default).

Then, run your migrations programmatically:

@ApplicationScoped
public class MigrationService {
    // You can Inject the object if you want to use it manually
    @Inject
    Flyway flyway; 

    public void checkMigration() {
        flyway.clean(); 
        flyway.migrate();
    }
}