Not able to read configuration from Consul in spring-boot application

Question

I am creating a Spring Boot application, which will read configuration like DB properties from Consul. But I am not able to read the key value from Consul using my application. Following is, what I am trying to do.

**pom.xml**
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.tuturself</groupId>
    <artifactId>spring-boot-consul</artifactId>
    <version>1.0-SNAPSHOT</version>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>1.5.4.RELEASE</version>
        <relativePath/>
    </parent>

    <properties>
        <java.version>1.8</java.version>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <spring.retry.version>1.2.1.RELEASE</spring.retry.version>
        <consul.version>1.1.2.RELEASE</consul.version>
        <consul.discovery.version>1.1.2.RELEASE</consul.discovery.version>
        <jackson.version>2.8.1</jackson.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-consul-all</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-consul-discovery</artifactId>
            <version>${consul.discovery.version}</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-config</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-aop</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.retry</groupId>
            <artifactId>spring-retry</artifactId>
            <version>${spring.retry.version}</version>
        </dependency>
        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-databind</artifactId>
            <version>${jackson.version}</version>
        </dependency>

        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-annotations</artifactId>
            <version>${jackson.version}</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.5.1</version>
                <configuration>
                    <source>1.8</source>
                    <target>1.8</target>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
                <executions>
                    <execution>
                        <goals>
                            <goal>repackage</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-consul-dependencies</artifactId>
                <version>1.2.1.RELEASE</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

</project>

And Following is my Main class:

@EnableRetry
@RefreshScope
@EnableDiscoveryClient
@SpringBootApplication
@ComponentScan("com.test.*")
public class SpringBootConsulApplication {

    private static ConsulConfiguration consulConfiguration;

    public static void main(String[] args) {
        try {
            String consulHost = System.getProperty("spring.cloud.consul.host");
            System.out.println("consulHost ::" + consulHost);
            String consulPort = System.getProperty("spring.cloud.consul.port");
            System.out.println("consulPort ::" + consulPort);
            String consulPrefix = System.getProperty("spring.cloud.consul.config.prefix");
            System.out.println("consulPrefix ::" + consulPrefix);
            new SpringApplicationBuilder(SpringBootConsulApplication.class).web(true).run(args);
        } catch (Exception ex) {
            ex.printStackTrace();
        }
    }
}

And I am reading the consul properties using the @Value annotation:

@Configuration
@EnableConfigurationProperties(PropertySourceBootstrapProperties.class)
public class ConsulConfiguration {

    @Value("${cassandra.host}")
    private String cassandraHost;

    @Value("${cassandra.user}")
    private String userName;

    @Value("${cassandra.password}")
    private String password;
}

I have my bootstrap.yml in resources folder:

spring:
  cloud:
    consul:
      host: localhost
      port: 8500
      enabled: true
      config:
        enabled: true
        prefix: config/application
        defaultContext: apps
        profileSeparator: '::'
  application:
    name: spring-boot-consul

Consul is up and running in my local system on localhost:8500 where I have the file config/application/spring-boot-consul.yml file;

spring:
  application:
    name: spring-boot-consul
cassandra:
  host: 127.0.0.1:9042,127.0.0.2:9042
  user: my_user
  password: my_pass
  pooling:
    maxThread: 10
    timeout: 50
  keyspace:
    name: test_keyspace
    readConsistency: ONE
    writeConsistency: ONE

When I am strating the application, it is showing not able to bind cassandra.host in my ConsulConfiguration class. Thus stopping the application. Any hints , What I am doing wrong here?

Answer 1

You can find a working example here.

Consul Configuration KV Store

You need to store your properties in the Consul KV store either from Consul UI or from the command line. The Consul Agent will not load your properties from the file system. To load the properties from the command line, you can use the following command once the Consul Agent is up and running. The YAML data can be read from a file by prefixing the file name with the @ symbol.

./consul kv put config/application/data @spring-boot-consul.yml

where config/application/data is the key name.

If the data is successfully written in the KV, you should get the following response,

Success! Data written to: config/application/data

You can also fetch the properties from the KV by using the following command,

$ ./consul kv get config/application/data
cassandra:
  host: 127.0.0.1:9042,127.0.0.2:9042
  user: my_user
  password: my_pass

You can also view the properties from the Consul Web UI,

Changes to bootstrap.yml

You need to modify your bootstrap.yml slightly. Here are the changes:

• prefix value to config
• defaultContext value to application
• Added format to yaml

• Added data-key by the name of data to fetch the YAML blob.

  spring:
    profiles: default
    cloud:
      consul:
        host: localhost
        port: 8500
        config:
          enabled: true
          prefix: config
          defaultContext: application
          data-key: data
          profileSeparator: '::'
          format: yaml
    application:
      name: spring-boot-consul
  

Changes to ConsulConfiguration

@Configuration
@RefreshScope
public class ConsulConfiguration {

    @Value("${cassandra.host}")
    private String cassandraHost;

    @Value("${cassandra.user}")
    private String userName;

    @Value("${cassandra.password}")
    private String password;

    @PostConstruct
    public void postConstruct() {
        // to validate if properties are loaded
        System.out.println("** cassandra.host: " + cassandraHost);
        System.out.println("** cassandra.user: " + userName);
        System.out.println("** cassandra.password: " + password);
    }
}

Changes to Application class,

@EnableRetry
@RefreshScope
@EnableDiscoveryClient
@EnableAutoConfiguration
@EnableConfigurationProperties
@SpringBootApplication
@ComponentScan("com.test.*")
public class SpringBootConsulApplication {

    public static void main(String[] args) {
        ...
    }

}