How to use different ports using Couchbase Java SDK 3

Question

Context:
I'm writing some integration tests using docker (using testcontainers). And I want to connect to couchbase using toxiproxy.

The problem:
I can't connect to couchbase using different ports than the default ones. The docs says to just use SeedNode, but it does not work.

Client:

<dependency>
    <groupId>com.couchbase.client</groupId>
    <artifactId>java-client</artifactId>
    <version>3.0.5</version>
</dependency>

Couchbase setup:

echo "Initializing new cluster"
docker-compose exec couchbase \
  couchbase-cli cluster-init -c 127.0.0.1 --cluster-username Administrator --cluster-password password \
  --services data,index,query --cluster-ramsize 4096

echo "Creating new bucket"
docker-compose exec couchbase \
  couchbase-cli bucket-create -c couchbase:8091 --username Administrator \
  --password password --bucket default --bucket-type couchbase \
  --bucket-ramsize 1024

It works:

services:
  couchbase:
    image: couchbase:latest
    volumes:
      - .couchbase:/opt/couchbase/var
    ports:
      - "8091:8091"
      - "11210:11210"

SeedNode seedNode = SeedNode.create("127.0.0.1", Optional.of(11210), Optional.of(8091));
ClusterOptions clusterOptions = ClusterOptions.clusterOptions("Administrator", "password");
Cluster cluster = Cluster.connect(Collections.singleton(seedNode), clusterOptions);

Bucket bucket = cluster.bucket("default");
Collection collection = bucket.defaultCollection();

collection.insert("key", "Hello, Couchbase!");

It doesn't work:

services:
  couchbase:
    image: couchbase:latest
    volumes:
      - .couchbase:/opt/couchbase/var
    ports:
      - "28091:8091"
      - "21210:11210"

SeedNode seedNode = SeedNode.create("127.0.0.1", Optional.of(21210), Optional.of(28091));

Error:
I see many warnings like:

[cb-events] WARN  com.couchbase.endpoint  - [com.couchbase.endpoint][EndpointConnectionFailedEvent][1282us] Connect attempt 7 failed because of AnnotatedConnectException: finishConnect(..) failed: Connection refused: /127.0.0.1:11210 {"bucket":"default","circuitBreaker":"DISABLED","coreId":"0xdf4c6e800000001","remote":"127.0.0.1:11210","type":"KV"}
com.couchbase.client.core.deps.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: /127.0.0.1:11210

And it fails on the insert line:

com.couchbase.client.core.error.AmbiguousTimeoutException: InsertRequest
 {"cancelled":true,"completed":true,"coreId":"0xdf4c6e800000001","idempotent":false,"reason":"TIMEOUT","requestId":3,"requestType":"InsertRequest","retried":14,"retryReasons":["ENDPOINT_NOT_AVAILABLE","BUCKET_OPEN_IN_PROGRESS"],"service":{"bucket":"default","collection":"_default","documentId":"key","opaque":"0x2","scope":"_default","syncDurability":{"present":false},"type":"kv"},"timeoutMs":2500,"timings":{"encodingMicros":5015,"totalMicros":2509788}}
    at com.couchbase.client.java.AsyncUtils.block(AsyncUtils.java:51)
    at com.couchbase.client.java.Collection.insert(Collection.java:381)
    at ToxyproxyCouchbaseIT.testCouchbase(ToxyproxyCouchbaseIT.java:85)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:686)
    at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
    at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
    at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
    at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
    at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
    at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
    at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
    at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
    at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
    at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
    at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
    at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
    at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:212)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:208)
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:137)
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:71)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:135)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
    at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
    at java.util.ArrayList.forEach(ArrayList.java:1257)
    at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
    at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
    at java.util.ArrayList.forEach(ArrayList.java:1257)
    at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
    at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
    at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
    at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
    at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
    at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:248)
    at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$5(DefaultLauncher.java:211)
    at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:226)
    at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:199)
    at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:132)
    at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:71)
    at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
    at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:220)
    at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:53)
    Suppressed: java.lang.Exception: The above exception was originally thrown by another thread at the following location.
        at com.couchbase.client.core.msg.BaseRequest.cancel(BaseRequest.java:163)
        at com.couchbase.client.core.Timer.lambda$register$2(Timer.java:157)
        at com.couchbase.client.core.deps.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:672)
        at com.couchbase.client.core.deps.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:747)
        at com.couchbase.client.core.deps.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:472)
        at com.couchbase.client.core.deps.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)

Somehow, using CouchbaseContainer instead of my own docker-compose it works with container's random ports, but it's not enough for me because I want to set toxiproxy ports instead.

Is it a bug or am I doing something wrong?

Answer 1

To answer the question (but not solve OP's problem, unfortunately), with SDK 3 you can specify a custom KV port like this:

 Cluster cluster = Cluster.connect("localhost:12345", username, password);

If you don't know the KV port, you can specify a manager port like this:

 Cluster cluster = Cluster.connect("localhost:45678=manager", username, password);

(As an alternative you can create SeedNode objects, but personally I find that a bit cumbersome.)

In order for custom ports to work, you will need to use Couchbase Server 6.5 or later which supports alternate addreses, and use the setting-alternate-address command to configure Couchbase to advertise the non-standard ports using "external" addresses.

The SDK uses the addresses passed to Cluster.connect to bootstrap the connections. Once it connects to the server, it throws away the addresses you gave it and instead uses the addresses it gets from the server. The server needs to be configured to report the "external" addresses so it can tell the client which hosts and ports to use for future requests.