Linux Image about azure-cosmos-db-emulator-docker

[dasjestyr]

Becoming increasingly problematic for modern development particularly when you're trying to develop hundreds of microservices on a mac. It's aggravating not being able to spin up a test instance in docker. Instead I have to go through the pain of getting that stuff set up correctly in Azure with all the naming conventions and such, burning more time and money.

[RohanNagar]

Any updates on this?

[anton-bruilo]

2022, June.. What about image for M1?

[sean-halpin]

Is this on the road map before 2025 ?

[garfbradaz]

Upvote here guys:

https://feedback.azure.com/forums/263030/suggestions/18533509

[koprivajakub]

M1 support is kind of must have for me. Do you have any timeline when this can be expected? arm64 is here for at least a year now. And not to mention that there won't be a single Mac without arm64 architecture in few years.

[dany74q]

+1 - Is this on the road map for 2020 ?

[atrauzzi]

I've mentioned this to Azure folks before, but I really think services shouldn't be released without a cross platform local emulation option.

I swear there's some merit in Azure creating a mini local cloud, complete with dashboard.

[giggio]

It would be nice for someone from the core team to answer. We need an ETA, even it is far away. Please don't leave us without an answer.

[dnwarnock]

You can just spin up a paired down test instance on Azure. I believe there is a free (or nearly free) tier available.

[nkzawa]

Hello, we just made a mock server implementation public. Hope it will be helpful for testing your applications.

https://github.com/zeit/cosmosdb-server

[daveHylde]

I've created this feedback post regarding arm64 support for the emulator (or docker image). Upvote if you agree.

https://feedback.azure.com/d365community/idea/dc98013b-7348-ed11-a81b-000d3ae3db6e

[theramzay]

September 2022 -- any news?

[jabba2324]

Its Feb 2021 and still no linux docker container..

[syedhassaanahmed]

+1 Any ETA on this?

[renato04]

Any updates on this?

[newtack]

The problem for us at this point is not for development in our case (we use Windows) but for CI/CD which runs on Docker (Linux). Can someone from Azure team comment on this?

[RohanNagar]

Looks like work on this has started today: https://feedback.azure.com/forums/263030-azure-cosmos-db/suggestions/18533509-add-documentdb-emulator-support-for-mac-os-x-and

[CharlieDigital]

@thesqlpro I disagree with the reasoning that it can't be done because it's hard to emulate.

• DynamoDB has a local downloadable runtime and a Docker image that works on Arm64 (We use this in CI/CD for integration tests and locally for development on M1 MacBook Pros when we don't need other AWS services)
• Google Firestore is part of the Firebase emulator suite and works great! (I've personally switched to Firestore for my own projects because it's so easy to use the emulator and Google Cloud Run makes working with .NET in a serverless manner very streamlined)
• LocalStack emulates most of the AWS stack (LocalStack isn't even first party AWS; I use this at work where we use it to emulate SNS/SQS locally, but it can emulate a whole swath of AWS services with high fidelity)

Let that sink in for a bit: you can have an API compatible AWS in a Docker container on your desktop against which you can build and experiment for free for the most part. You can run it in CI/CD as a target for deployment and integration tests. It even integrates with the AWS CLI tooling! I can literally test my CLI command scripts against LocalStack before testing in the cloud environment.

I am a part of this thread -- like many others -- because we love working on Azure and love some of the services on Azure (personally enjoy the DX of Azure Functions over Lambda), but these lingering issues just serve to highlight how much more work the Azure team has ahead of them to really win over developers and teams. Tooling is a big part of that and having this huge gap in tooling is puzzling. Microsoft won't even acknowledge it!

None of us here are asking if Microsoft can make something novel; they already have an emulator for CosmosDB! It already exists! They just need to make it compatible with an established CPU architecture. I don't understand the reasoning why Microsoft can't make this work or is choosing not to make this work. My guess is that the emulator or the core engine itself has some incompatible C/C++ dependencies which is why the x64 emulation doesn't work on Arm64.

That said, I recently came across an emulator built by Vercel of all companies.

https://github.com/vercel/cosmosdb-server

Not sure how well it works nor how compatible it is with the current CosmosDB API, but I would encourage folks to take a look.

[adammartin]

This is an extremely needed feature. At this point I regret choosing CosmosDB based on MS professional suggestion. The tooling just isn't there. It looks like a great platform but it's just missing support.

Please add Linux based version.

[pieterdv]

there's an early public preview available it seems: https://www.youtube.com/watch?v=NUfK6n8UXi8 and https://gist.github.com/StefArroyo/86a87056300fa58c2042e84cdfaa0e1c#file-azurecosmosdblinuxemulatorgist-md unfortunately not working on M1 macs, so not able to test myself, hope support for M1 is coming soon(ish)

[epomatti]

Upvote here guys:

https://feedback.azure.com/forums/263030/suggestions/18533509

Work started June 24, 2019. Go there and upvote.

[CharlieDigital]

@javierholguera unfortunately, it does not work (tried it when I first got my MBP M1) and that is what makes it puzzling from a technical perspective; I'm very curious why the emulation doesn't work for this image.

docker run --rm -it --platform linux/amd64 mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator
./palrun: ERROR: Invalid mapping of address 0x40037dd000 in reserved address space below 0x400000000000. Possible causes:
1) the process (itself, or via a wrapper) starts-up its own running environment sets the stack size limit to unlimited via syscall setrlimit(2);
2) the process (itself, or via a wrapper) adjusts its own execution domain and flag the system its legacy personality via syscall personality(2);
3) sysadmin deliberately sets the system to run on legacy VA layout mode by adjusting a sysctl knob vm.legacy_va_layout.

My workaround is to use a cloud instance of Cosmos in serverless mode ($0.38USD/mo so it's not a big deal), but it means I also need to have a good connection to work and that is not always the case.

[MrDizzle]

Any update for this please?

@MrDizzle When someone just asked two days ago, it's bad form to ask yet again right after, all you're doing is polluting people's notifications.

Wait isn't that what you are doing right now by chastising me? 😅

[newtack]

No, we do not use Azure DevOps. We use Docker (Linux) inside of Azure Container Instances. The reason we want to use the emulator is that we probably want to support thousands of requests per second but not have the overhead (network cost & monetary costs) and possible production interference by going directly against Cosmos. That is the key value proposition of the emulator and we want to leverage it inside of our Docker Linux instance.

[ltwlf]

Linux docker is needed. We use MAC and windows, every dev can choose what he prefer. On widows we use WSL/2. I don’t like the cloud option even there is a free tier. I want to use the same docker dev flow that I always use.

[Aravin]

Any update?

[D3MaxT]

Same problem. Windows 10 host, started a new app running in Docker Linux containers on WSL2. Wanted to try out Cosmos DB, so installed the emulator.

First run: "The remote certificate is invalid according to the validation procedure." error using https://host.docker.internal:8081 to connect.

Great development experience Microsoft. Just will have to use SQL again.

[milismsft]

@koprivajakub @pieterdv, please refer to https://docs.microsoft.com/en-us/azure/cosmos-db/linux-emulator?tabs=ssl-netstd21 for Cosmos DB related MacOS and Linux support.
We do not have a timeline currently regarding the Cosmos DB emulator running on M1/M1 Pro Apple silicon based chip. There are tough challenges ahead that might require a rewrite of significant parts of the emulator code, since currently our binaries are X64 based ones and require an Intel chip.

[newtack]

Looking for the same, any updated plans to support Linux?

[danielptm]

We need this. Our company is exploring using Mongodb atlas because they think it will in the grand scheme be cheaper because they will save on developer setup/troubleshooting time. If this were available for linux cosmos would have been the choice.

[CharlieDigital]

Understood, but there are other x64 images that work on M1 under emulation via Rosetta.

[bladelson]

Any update?

[Socolin]

Not sure how well it works nor how compatible it is with the current CosmosDB API, but I would encourage folks to take a look.

Last time I checked when using Linq it throw an error vercel/cosmosdb-server#59 and there is no commit since May, so I'm not sure this alternative is usable.
Also the linux image already available is very buggy (stop working after some times, with 1 cpu using 100%) , so don't get too much hope in the M1 version if MS don't address other problems, and so far I got no response to my bugs reports

[milismsft]

@JustinGrote the previous comment was published by mistake. Yes, we are currently working on adding Linux support for the Azure Cosmos DB emulator but we are in the very early stages of testing it out and by no means ready for the general public to try it out.
Please continue to monitor Microsoft official channels and we hope to make an announcement on this relatively soon.

[CharlieDigital]

@milismsft does the team have any insight into why the emulator image doesn't work on M1 Macs even with docker even with Rosetta? It's very unfortunate. I know that the serverless option is quite cheap for development use cases, but it would be nice to also have a local option to work with.

[garfbradaz]

@newtack: Are you using Azure Dev Ops? If so, does this help?

https://docs.microsoft.com/en-us/azure/cosmos-db/tutorial-setup-ci-cd

[rubit0]

No news? I honestly would be totally fine with a very crude emulator, heck it could be even based on MongoDb or something.

[jpbochi]

while there is no official emulator, does anyone know of any alternatives? Perhaps even some partial implementation.

[jpbochi]

You can just spin up a paired down test instance on Azure. I believe there is a free (or nearly free) tier available.

I'd like to avoid that. Obviously, it has lots of drawbacks, or a local DB would not even exist. Anyway, I'll have to investigate if that is a workable solution for my case. Thanks.

[JustinGrote]

Seems like the current plan is "go pound sand or use a live azure account".

CosmosDB does have a free tier available now, which helps, but it doesn't help for offline or air-gapped testing and development.

[codematrix]

It would be nice to at least create a cert like so: host.docker.internal

This way during local development, my Linux docker containers can at least hit https://host.docker.internal:8081 without complaining about "The SSL connection could not be established, see inner exception."

This would be a great plus.

[whung1]

@codematrix

To generate a cert, you can see this post where I talk about the steps
Azure/azure-cosmos-dotnet-v3#1232 (comment)

I would advise looking at the post later for updated scripts in case you want to automate this for your team.
Azure/azure-cosmos-dotnet-v3#1232 (comment)

Documentation is here on /GenCert and /FailOnSslMismatch flags:
https://docs.microsoft.com/en-us/azure/cosmos-db/local-emulator

[lonewarrior556]

@Azure Azure deleted a comment from thesqlpro 1 hour ago

What!?!?

[JustinGrote]

Seriously there was a comment from @thesqlpro that linked to (what I assume, did not test) was a linux docker container. Was that supposed to not be published yet? You're only going to Streisand effect it, I'm going to post it everywhere :P

[JustinGrote]

@milismsft thanks for clarifying, it is appreciated.

[thesqlpro]

Apologies for the mistake.

[TheOddler]

Please continue to monitor Microsoft official channels and we hope to make an announcement on this relatively soon.

Which official channels specifically? Has there been any announcements yet?

[JustinGrote]

@TheOddler https://devblogs.microsoft.com/cosmosdb/

[milismsft]

@CharlieDigital The Cosmos DB emulator Docker image for Linux requires a x64 compatible CPU. M1 is an arm64 based CPU.

[javierholguera]

@CharlieDigital would this approach work? https://forums.docker.com/t/run-x86-intel-and-arm-based-images-on-apple-silicon-m1-macs/117123

The default, of course, is to run the ARM version but if you use the --platform linux/amd64 parameter Docker will run the Intel version for you.

I don't have a Mac with an M1 to test but have been considering getting one and would be interested to see if this fixes it with Cosmos.

[BenjaBobs]

Also, any news on gremlin support?

[MrDizzle]

Any update for this please?

[JustinGrote]

Any update for this please?

@MrDizzle When someone just asked two days ago, it's bad form to ask yet again right after, all you're doing is polluting people's notifications.

[thesqlpro]

For now, if you can't get the emulator to work with the docker container then use the free tier:
https://learn.microsoft.com/en-us/azure/cosmos-db/free-tier

Cosmos DB is a PaaS service as such it is designed for the cloud. Hard to emulate all that on a personal computer, however I understand the need for testing/training/learning on something local. Not an answer, just a suggestion that might help on your Cosmos journey.