linuxdockerazure-cosmosdbazure-cosmosdb-emulator

Cosmos DB Docker Container Just Hangs when CosmosClient methods are called


I am having a lot of trouble using the Linux Cosmos DB Docker Container - mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator [latest].

UPDATE: The same issue was confirmed using the same container under Linux Mint and Windows - The host OS is not a factor.

I have managed to get it installed and running and I can access the local Emulator UI at https://localhost:8081/_explorer/index.html fine. From here I can create a DB and some containers and even add some random data.

The problems come in when I try to connect with code. I am connecting with the standard connection string...

AccountEndpoint=https://localhost:8081/;AccountKey=C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==

I initially hit some SSL issues which were resolved by importing the SSL certificate - This is a bit of a pain with a regenerating docker container but never mind. The main problem is that once I get past that any attempted operations just hang.

I started off trying to create the DB and containers, then I tried manually creating them and just trying to add data, then just loading some manually created items.

In all cases, as soon as the code calls into the CosmosClient methods such as...

all fail. I don't even get an error message, the code just hangs. It doesn't even time out - It will sit and spin for over an hour just doing nothing.

After some digging I found that the Debug Output window was just cycling with the following...

DocDBTrace Information: 0 : TimerPool Created with minSupportedTimerDelayInSeconds = 1
DocDBTrace Information: 0 : TimerPool Created with minSupportedTimerDelayInSeconds = 1
DocDBTrace Information: 0 : Resolving Master service address, forceMasterRefresh: False, currentMaster: 
DocDBTrace Warning: 0 : ClientRetryPolicy: Gateway HttpRequestException Endpoint not reachable. Failed Location: https://172.17.0.4:8081/; ResourceAddress: dbs/brain-store
DocDBTrace Information: 0 : GlobalEndpointManager: Marking endpoint https://172.17.0.4:8081/ unavailable for read
DocDBTrace Information: 0 : Current WriteEndpoints = (https://172.17.0.4:8081/) ReadEndpoints = (https://172.17.0.4:8081/)
DocDBTrace Information: 0 : Endpoint https://172.17.0.4:8081/ unavailable for Read added/updated to unavailableEndpoints with timestamp 04/22/2024 15:17:17
DocDBTrace Information: 0 : GlobalEndpointManager: Marking endpoint https://172.17.0.4:8081/ unavailable for Write
DocDBTrace Information: 0 : Current WriteEndpoints = (https://172.17.0.4:8081/) ReadEndpoints = (https://172.17.0.4:8081/)
DocDBTrace Information: 0 : Endpoint https://172.17.0.4:8081/ unavailable for Write added/updated to unavailableEndpoints with timestamp 04/22/2024 15:17:17
DocDBTrace Information: 0 : Current WriteEndpoints = (https://172.17.0.4:8081/) ReadEndpoints = (https://172.17.0.4:8081/)
DocDBTrace Information: 0 : Resolving Master service address, forceMasterRefresh: False, currentMaster: 
Exited Thread 94863
Exited Thread 95169
Exited Thread 95168
DocDBTrace Warning: 0 : ClientRetryPolicy: Gateway HttpRequestException Endpoint not reachable. Failed Location: https://172.17.0.4:8081/; ResourceAddress: dbs/brain-store
DocDBTrace Information: 0 : GlobalEndpointManager: Marking endpoint https://172.17.0.4:8081/ unavailable for read
DocDBTrace Information: 0 : Current WriteEndpoints = (https://172.17.0.4:8081/) ReadEndpoints = (https://172.17.0.4:8081/)
DocDBTrace Information: 0 : Endpoint https://172.17.0.4:8081/ unavailable for Read added/updated to unavailableEndpoints with timestamp 04/22/2024 15:17:28
DocDBTrace Information: 0 : GlobalEndpointManager: Marking endpoint https://172.17.0.4:8081/ unavailable for Write
DocDBTrace Information: 0 : Current WriteEndpoints = (https://172.17.0.4:8081/) ReadEndpoints = (https://172.17.0.4:8081/)
DocDBTrace Information: 0 : Endpoint https://172.17.0.4:8081/ unavailable for Write added/updated to unavailableEndpoints with timestamp 04/22/2024 15:17:28
DocDBTrace Information: 0 : Resolving Master service address, forceMasterRefresh: False, currentMaster: 
Exited Thread 95167
Exited Thread 94872
DocDBTrace Warning: 0 : ClientRetryPolicy: Gateway HttpRequestException Endpoint not reachable. Failed Location: https://172.17.0.4:8081/; ResourceAddress: dbs/brain-store
DocDBTrace Information: 0 : GlobalEndpointManager: Marking endpoint https://172.17.0.4:8081/ unavailable for read
DocDBTrace Information: 0 : Current WriteEndpoints = (https://172.17.0.4:8081/) ReadEndpoints = (https://172.17.0.4:8081/)
DocDBTrace Information: 0 : Endpoint https://172.17.0.4:8081/ unavailable for Read added/updated to unavailableEndpoints with timestamp 04/22/2024 15:17:38
DocDBTrace Information: 0 : GlobalEndpointManager: Marking endpoint https://172.17.0.4:8081/ unavailable for Write
DocDBTrace Information: 0 : Current WriteEndpoints = (https://172.17.0.4:8081/) ReadEndpoints = (https://172.17.0.4:8081/)
DocDBTrace Information: 0 : Endpoint https://172.17.0.4:8081/ unavailable for Write added/updated to unavailableEndpoints with timestamp 04/22/2024 15:17:38

(edited for brevity)

I have tried pinging 172.17.0.4 and it doesn't exist on my network, so I'm assuming it's only valid within the docker container itself - The issue being that, if that's the case, how is my code supposed to be able to connect to it? The errors suggest that the emulator is telling my code to connect to an endpoint that does not exist.

My docker container has the following settings:

Ports:

Am I missing something?


Solution

  • The solution to this was to make sure that the Docker container specifies an overriding IP Address - If you DON'T do this it seems that the container default is to use a random one.

    This can be resolved by setting the following argument parameter...

    AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE=127.0.0.1

    This made my full docker run command as...

    docker run -p 8081:8081 -p 10250:10250 -p 10251:10251 -p 10252:10252 -p 10253:10253 -p 10254:10254 -p 10255:10255 --env AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE=1 --env AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE=127.0.0.1 --name cosmos-emulator --pull missing -t -i mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:latest