Docker Colima + Testcontainers + Intellij

With Docker Desktop moving to a paid subscription, Colima being a lightweight, open-source container was a natural choice for many developers. This blog post focusses on changes that need to made to the local setup and in Intellij to ensure that the testcontainer setup works fine.

I) Installation and Setup

1. Install Colima

Install Colima and verify its installation:

brew install colima
colima --version

2. Start Colima virtual machine (VM)

colima start --cpu 2 --memory 4 --network-address <NETWORK_IP | 192.168.1.100>

The command starts a new Colima virtual machine (VM) with the specified CPU and memory resources. By default, Colima assigns a random IP address to the VM. This address can change each time you start the VM, making it difficult to configure static IP addresses or network rules.

By specifying a specific IP address with --network-address, you can ensure that the VM has a consistent IP address, making it easier to manage network configurations and access the VM from other devices.

Finally check the status of colima to see if you get the following output.

colima status

In the above image, we can see that the default docker socket has changed. This change in the network address and docker host(aka socket) needs to be reflected in the TestContainer setup.

3. Configure Environment Variables

The environment variables you are using (DOCKER_HOST, TESTCONTAINERS_HOST_OVERRIDE, and TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE) need to align with Colima's setup.

#  It should point to Colima's socket:
export DOCKER_HOST="unix://${HOME}/.colima/default/docker.sock"

# Testcontainers needs to know how to resolve the Colima host.
export TESTCONTAINERS_HOST_OVERRIDE=$(colima ls -j | jq -r '.address')

# For containers to communicate with Docker
export TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE=/var/run/docker.sock

Ensure these values are set in your shell or .bashrc/.zshrc and restart IntelliJ so that these enviroment variables are loaded in the Intellij context. Else, you would need to explicitly add these variables in the Test Configuration before running the tests.

Once the above steps are completed, we should be able to run the TestContainer tests.

II) Common Issues

1. Problem in determining docker configuration

Testcontainers might still struggle to determine the right Docker configuration. To solve this, try explicitly configuring the setup in your test class setup:

@Testcontainers
public class MyTests {

    static {
        System.setProperty("testcontainers.docker.client.strategy", "UnixSocketClientProviderStrategy");
    }
}

Check if colima has been selected as the default context for docker.

docker context ls

If not, then explicitly set docker to use colima

docker context use colima

Also, one needs to verify the IntelliJ Configuration to ensure it inherits the shell environment variables. Go to Run > Edit Configurations and under Environment Variables check if the following variables are inherited

DOCKER_HOST=unix:///Users/<your-user>/.colima/default/docker.sock
TESTCONTAINERS_HOST_OVERRIDE=<colima IP>
TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE=/var/run/docker.sock 

2. Corrupted Docker config

One reason for test containers failing to run might be the docker config getting corrupted. If that’s the case, running tests in Intellij would generate the following stack trace:

org.testcontainers.dockerclient.DockerClientProviderStrategy: Provider org.testcontainers.dockerclient.EnvironmentAndSystemPropertyClientProviderStrategy could not be instantiated
java.util.ServiceConfigurationError: org.testcontainers.dockerclient.DockerClientProviderStrategy: Provider org.testcontainers.dockerclient.EnvironmentAndSystemPropertyClientProviderStrategy could not be instantiated
 at java.base/java.util.ServiceLoader.fail(ServiceLoader.java:586)
 at java.base/java.util.ServiceLoader$ProviderImpl.newInstance(ServiceLoader.java:813)
 at java.base/java.util.ServiceLoader$ProviderImpl.get(ServiceLoader.java:729)
 at java.base/java.util.ServiceLoader$3.next(ServiceLoader.java:1403)
 at java.base/java.lang.Iterable.forEach(Iterable.java:74)
 at org.testcontainers.DockerClientFactory.getOrInitializeStrategy(DockerClientFactory.java:148)
 at org.testcontainers.DockerClientFactory.client(DockerClientFactory.java:186)
 at org.testcontainers.DockerClientFactory$1.getDockerClient(DockerClientFactory.java:104)
 at com.github.dockerjava.api.DockerClientDelegate.authConfig(DockerClientDelegate.java:109)
 at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:321)
 at org.testcontainers.junit.jupiter.TestcontainersExtension$StoreAdapter.start(TestcontainersExtension.java:274)
 at org.testcontainers.junit.jupiter.TestcontainersExtension$StoreAdapter.access$200(TestcontainersExtension.java:261)
 at org.testcontainers.junit.jupiter.TestcontainersExtension.lambda$null$3(TestcontainersExtension.java:76)
 at org.junit.jupiter.engine.execution.ExtensionValuesStore.lambda$getOrComputeIfAbsent$4(ExtensionValuesStore.java:86)
 at org.junit.jupiter.engine.execution.ExtensionValuesStore$MemoizingSupplier.computeValue(ExtensionValuesStore.java:223)
 at org.junit.jupiter.engine.execution.ExtensionValuesStore$MemoizingSupplier.get(ExtensionValuesStore.java:211)
 at org.junit.jupiter.engine.execution.ExtensionValuesStore$StoredValue.evaluate(ExtensionValuesStore.java:191)
 at org.junit.jupiter.engine.execution.ExtensionValuesStore$StoredValue.access$100(ExtensionValuesStore.java:171)
 at org.junit.jupiter.engine.execution.ExtensionValuesStore.getOrComputeIfAbsent(ExtensionValuesStore.java:89)
 at org.junit.jupiter.engine.execution.NamespaceAwareStore.getOrComputeIfAbsent(NamespaceAwareStore.java:53)
 at org.testcontainers.junit.jupiter.TestcontainersExtension.lambda$startContainers$4(TestcontainersExtension.java:76)
 at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
 at org.testcontainers.junit.jupiter.TestcontainersExtension.startContainers(TestcontainersExtension.java:76)
 at org.testcontainers.junit.jupiter.TestcontainersExtension.beforeAll(TestcontainersExtension.java:56)
 at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.lambda$invokeBeforeAllCallbacks$12(ClassBasedTestDescriptor.java:395)
 at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
 at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.invokeBeforeAllCallbacks(ClassBasedTestDescriptor.java:395)
 at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.before(ClassBasedTestDescriptor.java:211)
 at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.before(ClassBasedTestDescriptor.java:84)
 at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:148)
 at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
 at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
 at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
 at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
 at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
 at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
 at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
 at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
 at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
 at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
 at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
 at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
 at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
 at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
 at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
 at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
 at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
 at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35)
 at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
 at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54)
 at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:107)
 at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
 at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
 at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
 at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
 at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114)
 at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86)
 at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86)
 at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:53)
 at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:99)
 at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$000(JUnitPlatformTestClassProcessor.java:79)
 at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:75)
 at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:61)
 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
 at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.base/java.lang.reflect.Method.invoke(Method.java:568)
 at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
 at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
 at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
 at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
 at jdk.proxy1/jdk.proxy1.$Proxy2.stop(Unknown Source)
 at org.gradle.api.internal.tasks.testing.worker.TestWorker$3.run(TestWorker.java:193)
 at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129)
 at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100)
 at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60)
 at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
 at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:133)
 at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:71)
 at worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
 at worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)
 Suppressed: java.util.ServiceConfigurationError: org.testcontainers.dockerclient.DockerClientProviderStrategy: Provider org.testcontainers.dockerclient.EnvironmentAndSystemPropertyClientProviderStrategy could not be instantiated
  at java.base/java.util.ServiceLoader.fail(ServiceLoader.java:586)
  at java.base/java.util.ServiceLoader$ProviderImpl.newInstance(ServiceLoader.java:813)
  at java.base/java.util.ServiceLoader$ProviderImpl.get(ServiceLoader.java:729)
  at java.base/java.util.ServiceLoader$3.next(ServiceLoader.java:1403)
  at java.base/java.lang.Iterable.forEach(Iterable.java:74)
  at org.testcontainers.DockerClientFactory.getOrInitializeStrategy(DockerClientFactory.java:148)
  at org.testcontainers.DockerClientFactory.client(DockerClientFactory.java:186)
  at org.testcontainers.DockerClientFactory$1.getDockerClient(DockerClientFactory.java:104)
  at com.github.dockerjava.api.DockerClientDelegate.authConfig(DockerClientDelegate.java:109)
  at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:321)
  at org.testcontainers.junit.jupiter.TestcontainersExtension$StoreAdapter.start(TestcontainersExtension.java:274)
  at org.testcontainers.junit.jupiter.TestcontainersExtension$StoreAdapter.access$200(TestcontainersExtension.java:261)
  at org.testcontainers.junit.jupiter.TestcontainersExtension.lambda$null$3(TestcontainersExtension.java:76)
  at org.junit.jupiter.engine.execution.ExtensionValuesStore.lambda$getOrComputeIfAbsent$4(ExtensionValuesStore.java:86)
  at org.junit.jupiter.engine.execution.ExtensionValuesStore$MemoizingSupplier.computeValue(ExtensionValuesStore.java:223)
  at org.junit.jupiter.engine.execution.ExtensionValuesStore$MemoizingSupplier.get(ExtensionValuesStore.java:211)
  at org.junit.jupiter.engine.execution.ExtensionValuesStore$StoredValue.evaluate(ExtensionValuesStore.java:191)
  at org.junit.jupiter.engine.execution.ExtensionValuesStore$StoredValue.evaluateSafely(ExtensionValuesStore.java:183)
  at org.junit.jupiter.engine.execution.ExtensionValuesStore$StoredValue.access$400(ExtensionValuesStore.java:171)
  at org.junit.jupiter.engine.execution.ExtensionValuesStore.lambda$closeAllStoredCloseableValues$1(ExtensionValuesStore.java:65)
  at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:178)
  at java.base/java.util.concurrent.ConcurrentHashMap$ValueSpliterator.forEachRemaining(ConcurrentHashMap.java:3612)
  at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
  at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
  at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
  at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
  at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
  at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
  at org.junit.jupiter.engine.execution.ExtensionValuesStore.closeAllStoredCloseableValues(ExtensionValuesStore.java:68)
  at org.junit.jupiter.engine.descriptor.AbstractExtensionContext.close(AbstractExtensionContext.java:80)
  at org.junit.jupiter.engine.execution.JupiterEngineExecutionContext.close(JupiterEngineExecutionContext.java:53)
  at org.junit.jupiter.engine.descriptor.JupiterTestDescriptor.cleanUp(JupiterTestDescriptor.java:222)
  at org.junit.jupiter.engine.descriptor.JupiterTestDescriptor.cleanUp(JupiterTestDescriptor.java:57)
  at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$cleanUp$10(NodeTestTask.java:167)
  at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
  at org.junit.platform.engine.support.hierarchical.NodeTestTask.cleanUp(NodeTestTask.java:167)
  at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:98)
  ... 44 more
 Caused by: java.lang.IllegalArgumentException: Illegal base64 character a
  at java.base/java.util.Base64$Decoder.decode0(Base64.java:847)
  at java.base/java.util.Base64$Decoder.decode(Base64.java:566)
  at java.base/java.util.Base64$Decoder.decode(Base64.java:589)
  at org.testcontainers.shaded.com.github.dockerjava.core.DockerConfigFile.decodeAuth(DockerConfigFile.java:225)
  at org.testcontainers.shaded.com.github.dockerjava.core.DockerConfigFile.loadConfig(DockerConfigFile.java:160)
  at org.testcontainers.shaded.com.github.dockerjava.core.DefaultDockerClientConfig$Builder.readDockerConfig(DefaultDockerClientConfig.java:470)
  at org.testcontainers.shaded.com.github.dockerjava.core.DefaultDockerClientConfig$Builder.build(DefaultDockerClientConfig.java:457)
  at org.testcontainers.dockerclient.EnvironmentAndSystemPropertyClientProviderStrategy.<init>(EnvironmentAndSystemPropertyClientProviderStrategy.java:57)
  at org.testcontainers.dockerclient.EnvironmentAndSystemPropertyClientProviderStrategy.<init>(EnvironmentAndSystemPropertyClientProviderStrategy.java:34)
  at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
  at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
  at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)
  at java.base/java.util.ServiceLoader$ProviderImpl.newInstance(ServiceLoader.java:789)
  ... 79 more

Since the error points to Illegal base64 character, check for corruption in ~/.docker/config.json.

You can regenerate it by logging out and back in:

docker logout
docker login

or simply backing up and cleaning up the file:

mv ~/.docker/config.json ~/.docker/config.json.bak
docker login

This should fix the auth error in the docker config.

Conclusion

By following the steps outlined in this guide, you can successfully integrate Colima and Testcontainers into your IntelliJ development environment. Remember to configure environment variables correctly and address potential issues like corrupted Docker configurations. For detailed instructions and troubleshooting tips, refer to the official documentation for Colima and Testcontainers.

References

1. https://github.com/abiosoft/colima/
2. https://github.com/testcontainers/testcontainers-java/issues/5034
3. https://java.testcontainers.org/