vercel / cosmosdb-server Goto Github PK
View Code? Open in Web Editor NEWA Cosmos DB server implementation for testing your applications locally.
License: MIT License
A Cosmos DB server implementation for testing your applications locally.
License: MIT License
I've got no route
error when functions with patch operation like below was triggered.
container
.item('some_id')
.patch([{
op: 'add',
path: '/some_paths',
value: 'some_value'
}])
Patch operation is this
https://learn.microsoft.com/en-us/azure/cosmos-db/partial-document-update
Full response was like this
{
code: 400,
body: { message: 'no route' },
headers: {
'content-type': 'application/json',
'content-location': 'https://localhost:8081/dbs/next-auth/colls/users/docs/15b91394-2982-409f-8f71-024f4594872c',
connection: 'close',
'x-ms-activity-id': '8e4f9947-c8bc-4ba4-a624-42a169fbdb14',
'x-ms-request-charge': '1',
date: 'Wed, 01 Feb 2023 18:03:57 GMT',
'content-length': '22',
'x-ms-throttle-retry-count': 0,
'x-ms-throttle-retry-wait-time-ms': 0
},
activityId: '8e4f9947-c8bc-4ba4-a624-42a169fbdb14'
}
Hello,
I'm trying to create a trigger, but is returning a bad request.
Here it is my code
from pydocumentdb import document_client
DB_DATABASE = 'test_database2'
DB_COLLECTION = 'my_collection'
client = document_client.DocumentClient(
'https://localhost:3000/', {'masterKey': ''})
print("deleting database")
try:
db = next((data for data in client.ReadDatabases(
) if data['id'] == DB_DATABASE))
client.DeleteDatabase(db['_self'])
except:
pass
print("creating database")
db = client.CreateDatabase({'id': DB_DATABASE})
print("creating collection")
collection = client.CreateCollection(db['_self'], {'id': DB_COLLECTION})
path = 'dbs/{}/colls/{}'.format(DB_DATABASE, DB_COLLECTION)
# Create trigger
trigger_definition = {
'id': '_type',
'serverScript': """
// Ensures that documents have a _type field.
function validateNameExists() {
var collection = getContext().getCollection();
var request = getContext().getRequest();
var docToCreate = request.getBody();
// Reject documents that do not have a name property by throwing an exception.
if (!docToCreate._type) {
throw new Error('Document must include a "_type" property.');
}
}
""",
'triggerType': 'Pre',
'triggerOperation': 'All'
}
try:
client.CreateTrigger(collection['_self'], trigger_definition)
except Exception as e:
print(e)
The request is created for
'https://localhost:3000//dbs/wgLLAG--C30lC30lC30lD0--/colls/V2u_QG--C30lC30lC30lC0--/triggers/'
With this header
<class 'dict'>: {'Cache-Control': 'no-cache', 'x-ms-version': '2016-07-11', 'User-Agent': 'Windows/10 Python/3.6.8 documentdb-python-sdk/2.0.0', 'x-ms-consistency-level': 'Session', 'x-ms-session-token': '', 'Content-Type': 'application/json', 'Accept': 'application/json'}
And with this body
<class 'dict'>: {'id': '_type', 'triggerType': 'Pre', 'triggerOperation': 'All', 'body': '\n // Ensures that documents have a _type field.\n function validateNameExists() {\n var collection = getContext().getCollection();\n var request = getContext().getRequest();\n var docToCreate = request.getBody();\n\n // Reject documents that do not have a name property by throwing an exception.\n if (!docToCreate._type) {\n throw new Error(\'Document must include a "_type" property.\');\n }\n }\n '}```
Seems java client is stricter than node one and doesn't work properly.
Maybe we want to add integration tests with the client and ensure it works.
This is a SQL query I'm looking at:
SELECT * FROM c WHERE c.foo.bar = "xyz"
In this collection we have some documents with foo
field and for some other's it's undefined
.
Then when we ran this query, client give us an error and the server reports this:
TypeError: Cannot read property 'bar' of undefined
at $h.paginate.$h.sort.$c.reduce.filter (eval at exports.default (/data/zeit/api/services/cosmos-local/node_modules/@zeit/cosmosdb-query/lib/executor.js:8:21), <anonymous>:8:83)
at Array.filter (<anonymous>)
at eval (eval at exports.default (/data/zeit/api/services/cosmos-local/node_modules/@zeit/cosmosdb-query/lib/executor.js:8:21), <anonymous>:6:15)
at Object.exports.default (/data/zeit/api/services/cosmos-local/node_modules/@zeit/cosmosdb-query/lib/executor.js:13:12)
at Query.exec (/data/zeit/api/services/cosmos-local/node_modules/@zeit/cosmosdb-query/lib/index.js:29:34)
at Documents.query (/data/zeit/api/services/cosmos-local/node_modules/@zeit/cosmosdb-server/lib/account/items.js:53:55)
at _read_items_1.default (/data/zeit/api/services/cosmos-local/node_modules/@zeit/cosmosdb-server/lib/handler/query-documents.js:24:33)
at process._tickCallback (internal/process/next_tick.js:68:7)
I am trying to use the cosmosdb-server for testing an ASP.net core application, but I always get an exception when trying to interact with the server:
System.Net.Http.HttpRequestException : The SSL connection could not be established, see inner exception. ---- System.Security.Authentication.AuthenticationException : The remote certificate is invalid according to the validation procedure.
Did I do something wrong?
I did not find a way to disable SSL certificate verification for CosmosClient.
I know, that the older DocumentClient has an option to pass a custom handler to ignore the SSL certificate, but there does not seem to be such an option for the CosmosClient.
Hi!
When i use test key
as my dummy key I'm getting encode error, but when i use a valid key like emulator key the connection is refused.
I'm using python to test the connection.
Thanks in advance
The code Lib I am using makes a query to
//addresses/?$resolveFor=dbs&$filter=protocol%20eq%20rntbd
when it launches..
it expects Json looking like this:
{
"Addresss":[
{
"isPrimary":true,
"physcialUri":"rntbd://foo.documents.azure.com:11018/apps/16d6d56e-e73d-4069-98f5-2e677328a7d1/services/ce59e2ac-eac5-490f-9262-585b250f0399/partitions/2fe30e96-a74a-4596-a674-2c04a39c8603/replicas/132312777763206824p/",
"protocol":"rntbd",
"partitionKeyRangeId":"M",
"partitionIndex":"0@0"
},
{
"isPrimary":false,
"physcialUri":"rntbd://foo.documents.azure.com:11316/apps/16d6d56e-e73d-4069-98f5-2e677328a7d1/services/ce59e2ac-eac5-490f-9262-585b250f0399/partitions/2fe30e96-a74a-4596-a674-2c04a39c8603/replicas/132312759686602427s/",
"protocol":"rntbd",
"partitionKeyRangeId":"M",
"partitionIndex":"0@0"
},
{
"isPrimary":false,
"physcialUri":"rntbd://foo.documents.azure.com:11011/apps/16d6d56e-e73d-4069-98f5-2e677328a7d1/services/ce59e2ac-eac5-490f-9262-585b250f0399/partitions/2fe30e96-a74a-4596-a674-2c04a39c8603/replicas/132312759686602428s/",
"protocol":"rntbd",
"partitionKeyRangeId":"M",
"partitionIndex":"0@0"
},
{
"isPrimary":false,
"physcialUri":"rntbd://foo.documents.azure.com:11009/apps/16d6d56e-e73d-4069-98f5-2e677328a7d1/services/ce59e2ac-eac5-490f-9262-585b250f0399/partitions/2fe30e96-a74a-4596-a674-2c04a39c8603/replicas/132312804907485886s/",
"protocol":"rntbd",
"partitionKeyRangeId":"M",
"partitionIndex":"0@0"
}
],
"_count":4
}
without the $filter=protocol%20eq%20rntbd
it includes https ie
{"isPrimary":true,
"physcialUri":"https://foo.documents.azure.com:11618/apps/16d6d56e-e73d-4069-98f5-2e677328a7d1/services/ce59e2ac-eac5-490f-9262-585b250f0399/partitions/2fe30e96-a74a-4596-a674-2c04a39c8603/replicas/132312777763206824p//",
"protocol":"https",
"partitionKeyRangeId":"M",
"partitionIndex":"0@0"}
first of all wth is rntbd protocol, can it be mocked?
But the issue is that with out //addresses/blah
returning anything, The Lib crashes parsing non existent json.
Can this be added to routes?
Hello!
I run npm install
.
And now I'm trying to use cosmosdb-server -p 3000
but is giving command not found error
Am I doing something wrong?
Thanks!
Hi,
we try to integrate your project within our CI/CD build pipeline. One test uses an Upsert command (2nd) to update an existing entry, which was previously added with a first upsert command (1st).
When the 2nd Upsert statement is issued, we get this reply:
{HTTPFailure}Status code: 409
{"code":"Conflict","message":"Resource with specified id or name already exists."}
As far as I understand, Upsert should NEVER return an error like "already exists" since the entry should just be overwritten or inserted. The test runs fine when running it agains a local installation of the Azure CosmosDB Emulator (https://docs.microsoft.com/de-de/azure/cosmos-db/local-emulator).
I guess this is a bug?
I'm trying to use cosmosdb-server with the NuGet package Microsoft.EntityFrameworkCore.Cosmos version 3.1.0 and get a NullReferenceException
during application startup when calling .Database.EnsureCreated()
:
crit: Microsoft.AspNetCore.Hosting.Diagnostics[6]
Application startup exception
System.NullReferenceException: Object reference not set to an instance of an object.
at Microsoft.Azure.Cosmos.CosmosAccountServiceConfiguration.get_DefaultConsistencyLevel()
at Microsoft.Azure.Cosmos.DocumentClient.GetInitializationTaskAsync(IStoreClientFactory storeClientFactory)
at Microsoft.Azure.Cosmos.DocumentClient.EnsureValidClientAsync()
at Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler.EnsureValidClientAsync(RequestMessage request)
at Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler.SendAsync(RequestMessage request, CancellationToken cancellationToken)
at Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler.SendAsync(Uri resourceUri, ResourceType resourceType, OperationType operationType, RequestOptions requestOptions, ContainerCore cosmosContainerCore, Nullable`1 partitionKey, Stream streamPayload, Action`1 requestEnricher, CancellationToken cancellationToken)
at Microsoft.Azure.Cosmos.CosmosClient.CreateDatabaseIfNotExistsAsync(String id, Nullable`1 throughput, RequestOptions requestOptions, CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.CosmosClientWrapper.CreateDatabaseIfNotExistsOnceAsync(DbContext _, Object __, CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.CosmosClientWrapper.CreateDatabaseIfNotExistsOnce(DbContext context, Object state)
at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.ExecuteImplementation[TState,TResult](Func`3 operation, Func`3 verifySucceeded, TState state)
at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.Execute[TState,TResult](TState state, Func`3 operation, Func`3 verifySucceeded)
at Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.CosmosClientWrapper.CreateDatabaseIfNotExists()
at Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.CosmosDatabaseCreator.EnsureCreated()
at Microsoft.EntityFrameworkCore.Infrastructure.DatabaseFacade.EnsureCreated()
at testclient.Startup.Configure(IApplicationBuilder app, IWebHostEnvironment env, TestContext dbContext) in /home/martin/projects/cosmosdb-server/testclient/Startup.cs:line 34
at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at Microsoft.AspNetCore.Hosting.ConfigureBuilder.Invoke(Object instance, IApplicationBuilder builder)
at Microsoft.AspNetCore.Hosting.ConfigureBuilder.<>c__DisplayClass4_0.<Build>b__0(IApplicationBuilder builder)
at Microsoft.AspNetCore.Hosting.GenericWebHostBuilder.<>c__DisplayClass13_0.<UseStartup>b__2(IApplicationBuilder app)
at Microsoft.AspNetCore.HostFilteringStartupFilter.<>c__DisplayClass0_0.<Configure>b__0(IApplicationBuilder app)
at Microsoft.AspNetCore.Hosting.GenericWebHostService.StartAsync(CancellationToken cancellationToken)
Unhandled exception. System.NullReferenceException: Object reference not set to an instance of an object.
at Microsoft.Azure.Cosmos.CosmosAccountServiceConfiguration.get_DefaultConsistencyLevel()
at Microsoft.Azure.Cosmos.DocumentClient.GetInitializationTaskAsync(IStoreClientFactory storeClientFactory)
at Microsoft.Azure.Cosmos.DocumentClient.EnsureValidClientAsync()
at Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler.EnsureValidClientAsync(RequestMessage request)
at Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler.SendAsync(RequestMessage request, CancellationToken cancellationToken)
at Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler.SendAsync(Uri resourceUri, ResourceType resourceType, OperationType operationType, RequestOptions requestOptions, ContainerCore cosmosContainerCore, Nullable`1 partitionKey, Stream streamPayload, Action`1 requestEnricher, CancellationToken cancellationToken)
at Microsoft.Azure.Cosmos.CosmosClient.CreateDatabaseIfNotExistsAsync(String id, Nullable`1 throughput, RequestOptions requestOptions, CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.CosmosClientWrapper.CreateDatabaseIfNotExistsOnceAsync(DbContext _, Object __, CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.CosmosClientWrapper.CreateDatabaseIfNotExistsOnce(DbContext context, Object state)
at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.ExecuteImplementation[TState,TResult](Func`3 operation, Func`3 verifySucceeded, TState state)
at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.Execute[TState,TResult](TState state, Func`3 operation, Func`3 verifySucceeded)
at Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.CosmosClientWrapper.CreateDatabaseIfNotExists()
at Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.CosmosDatabaseCreator.EnsureCreated()
at Microsoft.EntityFrameworkCore.Infrastructure.DatabaseFacade.EnsureCreated()
at testclient.Startup.Configure(IApplicationBuilder app, IWebHostEnvironment env, TestContext dbContext) in /home/martin/projects/cosmosdb-server/testclient/Startup.cs:line 34
at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at Microsoft.AspNetCore.Hosting.ConfigureBuilder.Invoke(Object instance, IApplicationBuilder builder)
at Microsoft.AspNetCore.Hosting.ConfigureBuilder.<>c__DisplayClass4_0.<Build>b__0(IApplicationBuilder builder)
at Microsoft.AspNetCore.Hosting.GenericWebHostBuilder.<>c__DisplayClass13_0.<UseStartup>b__2(IApplicationBuilder app)
at Microsoft.AspNetCore.HostFilteringStartupFilter.<>c__DisplayClass0_0.<Configure>b__0(IApplicationBuilder app)
at Microsoft.AspNetCore.Hosting.GenericWebHostService.StartAsync(CancellationToken cancellationToken)
at Microsoft.Extensions.Hosting.Internal.Host.StartAsync(CancellationToken cancellationToken)
at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.Run(IHost host)
at testclient.Program.Main(String[] args) in /home/martin/projects/cosmosdb-server/testclient/Program.cs:line 16
It seems to expect some data about consistency levels being returned from Cosmos.
Minimal example can be created by:
dotnet new web
public class TestContext : DbContext {
public TestContext(DbContextOptions options) : base(options){}
}
public class Startup
{
public void ConfigureServices(IServiceCollection services) {
services.AddDbContext<TestContext>(opt =>
{
opt.UseCosmos("https://localhost:8081", "ZHVtbXk=", "some-db");
});
}
public void Configure(TestContext dbContext) {
dbContext.Database.EnsureCreated();
}
}
dotnet run
The C# SDK currently does not work with this CosmosDb server.
NuGet package: https://www.nuget.org/packages/Microsoft.Azure.Cosmos/
I discovered the following issues while attempting to make it work:
message
property with non-null value in the response body. Otherwise it will throw a NullReferenceException.queryEngineConfiguration
property is required for querying to work.partitionKeyRanges
is invalid, it is maxInclusive
but should be maxExclusive
"True"
value instead of "true"
, causing the server to not handle them as trueA-IM
header. Essentially it expects an etag when retrieving partition keys, and it expects a 302 status code once all keys have been received. Current behavior causes an infinite loop in the C# SDKAfter forking this repo and applying fixes for the above I am able to successfully use this server with the C# SDK.
I noticed you have a current draft PR with some fixes for the Java SDK, which will fix some of the issues here. I can make a PR with the remaining fixes if you'd like
New features were recently added and are missing from this library.
Hi!
After I run the test I want to see the document created.
Is there a way to do that?
I'm accessing https://localhost:3000
and I'm getting {}
as a response.
When creating a function app and a function with cosmosdb input binding and a sqlquery,
two post requests are made to "/dbs/:dbId/colls/:collId/docs", one of which has the "x-ms-cosmos-is-query-plan-request: True" header.
The Azure function expects a specific response from this request, after which the actual query request will be made.
The expected resposne looks like this:
{
"partitionedQueryExecutionInfoVersion": "2",
"queryInfo": {
"distinctType": "None",
"top": null,
"offset": null,
"limit": null,
"orderBy": [],
"orderByExpressions": [],
"groupByExpressions": [],
"groupByAliases": [],
"aggregates": [],
"groupByAliasToAggregateType": {},
"rewrittenQuery": "",
"hasSelectValue": false,
"dCountInfo": null
},
"queryRanges": [
{
"min": "05C1A3C5D33F20083200",
"max": "05C1A3C5D33F20083200",
"isMinInclusive": true,
"isMaxInclusive": true
}
]
}
with attribute values depending on the specific query, for instance if "order by" is added, the attributes
orderBy and orderByExpressions will contain "Ascending/Descending" and the column names respectively
This is because the hostname in databaseAccountEndpoint is set to the hostname at startup, however
accessing another service in a docker-compose project requires one to connect to the service name as hostname.
This can be fixed by setting databaseAccountEndpoint to the host in the requests header
Self explanatory,
when creating a simple azure function app with a cosmosDB input and adding a connection string in environment variables,
despite the connection string refering to "http://..." it will always try to connect to "https://...".
This is because databaseAccountEndpoint is hardcoded to be https
Hi guys. Thanks for this repo. I'm now using it succesfully to mock CosmosDb for the Azure IoT Device Simulation solution (https://github.com/Azure/device-simulation-dotnet/tree/DS-2.0.6).
I've set it up with docker-compose and have generated a self-signed CA certificate for the assigned hostname, and added it to the device simulation container, so when it connects using the Azure CSharp SDK, all SSL checks out and it just works(tm), aside from a few quirks that I had to look at (#46 and #48).
In doing this, I found a need for two main things:
My solution was to take the index.js (after compilation) and provide a replacement instead, packaging it with the docker image.
I'm not a node wiz, by far, so my logging additions have just been hacked together from what I could find from examples online, here it is: https://gist.github.com/dsschneidermann/a9314afee146feaffbeada42829e2e73
I can also provide the Dockerfile, docker-compose.yml etc if anyone is interested.
Edit: I forgot, I'm also overriding the read-meta.js file in order to set the self-aware "databaseAccountEndpoint" address and switch the default consistency level to "Strong", as it's needed for my case. Gist: https://gist.github.com/dsschneidermann/952f9160d5ce08aa69908b0862dcb5c6
Are there any plans to add gremlin/graph api support to this emulator?
Hello, members of Zeit
With the official CosmosDB emulator not running on non-windows, this is more or less exactly what I'm looking for. I was however not able to use it together with the Java SDK.
It seems like the Node SDK is quite lenient and performs little to no validation on the resource ids*. Hence, generating RID's like you do probably work (I haven't actually tested your implementation with the Node SDK, but I assume it does given the readme etc).
The Java SDK however is a bit more strict and assigns much more meaning to them, as shown here. In addition to be accepted by the parser, they also need to successfully serialize back to the original string.
Since your implementation only generates a quite random ID (of 14 bytes?), the Java SDK will fail on inserting documents, as shown below.
java.lang.IllegalArgumentException: Invalid resource id PEtvD0--C30lC30lC30lC0--
at com.microsoft.azure.documentdb.internal.ResourceId.parse(ResourceId.java:57)
at com.microsoft.azure.documentdb.internal.routing.CollectionCache.resolveByRid(CollectionCache.java:116)
at com.microsoft.azure.documentdb.internal.routing.CollectionCache.resolveCollection(CollectionCache.java:49)
at com.microsoft.azure.documentdb.internal.SessionContainer.resolvePartitionKeyRange(SessionContainer.java:217)
at com.microsoft.azure.documentdb.internal.SessionContainer.resolveSessionToken(SessionContainer.java:123)
at com.microsoft.azure.documentdb.DocumentClient.applySessionToken(DocumentClient.java:3222)
at com.microsoft.azure.documentdb.DocumentClient.doQuery(DocumentClient.java:3143)
at com.microsoft.azure.documentdb.DocumentQueryClientInternal.doQuery(DocumentQueryClientInternal.java:47)
at com.microsoft.azure.documentdb.internal.query.AbstractQueryExecutionContext.executeRequest(AbstractQueryExecutionContext.java:219)
at com.microsoft.azure.documentdb.internal.query.DefaultQueryExecutionContext.executeOnce(DefaultQueryExecutionContext.java:159)
at com.microsoft.azure.documentdb.internal.query.DefaultQueryExecutionContext.fillBuffer(DefaultQueryExecutionContext.java:99)
at com.microsoft.azure.documentdb.internal.query.DefaultQueryExecutionContext.hasNext(DefaultQueryExecutionContext.java:71)
at com.microsoft.azure.documentdb.internal.query.ProxyQueryExecutionContext.<init>(ProxyQueryExecutionContext.java:67)
at com.microsoft.azure.documentdb.internal.query.QueryExecutionContextFactory.createQueryExecutionContext(QueryExecutionContextFactory.java:23)
at com.microsoft.azure.documentdb.QueryIterable.createQueryExecutionContext(QueryIterable.java:70)
at com.microsoft.azure.documentdb.QueryIterable.reset(QueryIterable.java:115)
at com.microsoft.azure.documentdb.QueryIterable.<init>(QueryIterable.java:57)
at com.microsoft.azure.documentdb.DocumentClient.queryDocuments(DocumentClient.java:1167)
at com.microsoft.azure.documentdb.DocumentClient.queryDocuments(DocumentClient.java:1138)
* The source linked is of a quite old version. I can't seem to find the source for the most recent version of the Java SDK anywhere, reported here. This is obviously quite unfortunate, who knows what the SDK is doing these days.
Having trouble with GROUP BY
in a query.
const { resources } = await myContainer.items
.query({
query:
'SELECT MAX(t.inc) AS inc FROM t WHERE t.userId=@userId GROUP BY t.orgId',
parameters: [
{
name: '@userId',
value: userId,
},
],
})
.fetchAll();
The above works just fine using a real Cosmos NoSQL server, but errors with this one with the following...
Error: Expected "!=", "%", "&", "*", "+", "-", "--", ".", "/", "<", "<<", "<=", "<>", "=", ">", ">=", ">>", ">>>", "?", "??", "AND", "BETWEEN", "IN", "OR", "ORDER", "[", "^", "|", "||", [ \t\n\r], or end of input but "G" found.
"@vercel/cosmosdb-server": "^0.14.0",
"@azure/cosmos": "^3.17.3",
Node: 18.12.1
Thanks
Any interest in publishing a docker image for this? I'm using this Dockerfile in an integration testing story via docker-compose and it appears to be working great
Hello,
I started the server that way:
$ node lib/cli.js -p 8081
Ready to accept connections at [::]:8081
Then I installed the certificates so connection does not fail due to ssl error (If you are interesting I could write a PR for that, (the current cert.pem
does not seems valid)
Then, when I try to connect to Cosmos DB using dotnet sdk, it's stuck
private const string ConnectionString = "AccountEndpoint=https://localhost:8081/;AccountKey=C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==";
var cosmosClient = new CosmosClientBuilder(ConnectionString)
.Build();
So I ran it in with the debugger and notice those messages spamming in the debug console I can see it try to connect to another port
DocDBTrace Information: 0 : Marking endpoint https://localhost:3000/ unavailable for read
DocDBTrace Information: 0 : Current WriteEndpoints = (https://localhost:3000/) ReadEndpoints = (https://localhost:3000/)
DocDBTrace Information: 0 : Endpoint https://localhost:3000/ unavailable for Read added/updated to unavailableEndpoints with timestamp 12/20/2020 19:29:21
DocDBTrace Information: 0 : Resolving Master service address, forceMasterRefresh: False, currentMaster:
This seems to be an hardcoded value from https://github.com/vercel/cosmosdb-server/blob/master/src/handler/read-meta.ts#L8
Input query (raw, untransformed):
SELECT
AVG(c.name),
COUNT(1) AS deploys
FROM (
SELECT VALUE 1
) AS c
Hi,
I'm able to properly start the app (after installing with npm -g):
$ cosmosdb-server -p 3000
Ready to accept connections at [::]:3000
But I can't hit it:
$ curl localhost:3000
curl: (52) Empty reply from server
I'm on node v12.13.0 on RHEL7.
Do you know how I can proceed with this?
It appears that the delete-database-endpoint returns nothing (empty response), which is unexpected (at least to the Node SDK). Test case to reproduce shown below.
--- a/test/index.ts
+++ b/test/index.ts
@@ -97,3 +97,9 @@ export const udf = withTestEnv(async client => {
.fetchAll();
assert.deepStrictEqual(resources, [true]);
});
+
+export const deleteDatabase = withTestEnv(async client => {
+ await client.databases.create({ id: "test-database" });
+
+ await assert.doesNotReject(() => client.database("test-database").delete());
+});
This is from working on compatibility with the CosmosDb CSharp SDK
Error logged request:
{
"method":"GET",
"url":"/dbs/devicesimulation/colls/simulations/docs",
"docdbHeaders":[
"x-ms-documentdb-isquery",
"True",
"x-ms-documentdb-query-enablecrosspartition",
"False",
"x-ms-documentdb-query-iscontinuationexpected",
"False"
],
"response":{
"statusCode":400,
"statusMessage":"Bad Request",
"headers":{
"content-type":"application/json",
"content-location":"https://cosmosdb-sim:3000/dbs/devicesimulation/colls/simulations/docs",
"connection":"close",
"x-ms-activity-id":"5879c77f-a5fe-432a-a4d2-4d9454658713",
"x-ms-request-charge":"1"
},
"body":{
"message":"no route"
}
},
"errorMessage":null
}```
Expected result:
* Document is returned
Cause:
* Route lookup appends "_QUERY" when it sees that `x-ms-documentdb-isquery` is set.
Proposed fix by the associated PR:
* Restrict route lookup to only add _QUERY and _UPSERT for POST, where they are applicable
This is from working on compatibility with the CosmosDb CSharp SDK
Error logged request:
output for read-document.ts: getCollectionPartitionKeys(collection: ItemObject):
collection: {
geospatialConfig: {},
id: 'main',
indexingPolicy: {
automatic: true,
indexingMode: 'consistent',
includedPaths: [ [Object] ],
excludedPaths: [],
compositeIndexes: [],
spatialIndexes: []
},
_etag: '96751c73-738f-46df-aa37-54aa6dc133b5',
_rid: 'gAABAIAAAAM=',
_self: '/dbs/gAABAA==/colls/gAABAIAAAAM=',
_ts: 1604120454
}
TypeError: Cannot read property 'paths' of undefined
at getCollectionPartitionKeys (/app/node_modules/@zeit/cosmosdb-server/lib/handler/read-document.js:15:36)
at exports.default (/app/node_modules/@zeit/cosmosdb-server/lib/handler/read-document.js:61:32)
at /app/node_modules/run-cosmosdb-sim/index.js:97:34
at Server.<anonymous> (/app/node_modules/run-cosmosdb-sim/index.js:131:11)
at Server.emit (events.js:314:20)
at parserOnIncoming (_http_server.js:779:12)
at HTTPParser.parserOnHeadersComplete (_http_common.js:122:17)
{
"method":"GET",
"url":"/dbs/devicesimulation/colls/main/docs/MasterNode",
"docdbHeaders":[
"x-ms-documentdb-partitionkey",
"[]"
],
"response":{
"statusCode":500,
"statusMessage":"Internal Server Error",
"headers":{
"content-type":"application/json",
"content-location":"https://cosmosdb-sim:3000/dbs/devicesimulation/colls/main/docs/MasterNode",
"connection":"close",
"x-ms-activity-id":"7ac85c27-a88b-4099-bb94-e405df69f721",
"x-ms-request-charge":"1"
},
"body":{
"message":"Cannot read property 'paths' of undefined"
}
},
"errorMessage":null
}
Expected result:
Cause:
Proposed fix by this PR:
I can't seem to find any detailed license information within this project. The package.json states that the license is MIT, but it would be good for confidence if this was corroborated by a LICENSE file in the project root. Would you mind adding one, or accepting one via a PR?
Hi,
The route "/dbs/:dbId/colls/:collId/docs": queryDocuments,
is not responding the expected body when using the following example with dotnet sdk
var iterator = container.GetItemLinqQueryable<SomeObject>()
.Where(x => ids.Contains(x.Id))
.ToFeedIterator();
The responses received is (https://github.com/Azure/azure-cosmos-dotnet-v3/blob/master/Microsoft.Azure.Cosmos/src/Query/Core/Pipeline/CosmosQueryExecutionContextFactory.cs#L217)
{"Documents":[{"id":"1","someData":["a"],"_etag":"93f16095-8d53-4dc8-b8b7-11b9c29d0d60","_rid":"gAAGAIAAAAEBAAAAAAAAAA==","_self":"/dbs/gAAGAA==/colls/gAAGAIAAAAE=/docs/gAAGAIAAAAEBAAAAAAAAAA==/","_ts":1608932852},{"id":"2","someData":["b"],"_etag":"842c8917-e84b-4a92-99e8-4237daa2ff85","_rid":"gAAGAIAAAAECAAAAAAAAAA==","_self":"/dbs/gAAGAA==/colls/gAAGAIAAAAE=/docs/gAAGAIAAAAECAAAAAAAAAA==/","_ts":1608932852},{"id":"3","someData":["c"],"_etag":"e8033da5-1761-48a7-b0dd-365fe1548a0b","_rid":"gAAGAIAAAAEDAAAAAAAAAA==","_self":"/dbs/gAAGAA==/colls/gAAGAIAAAAE=/docs/gAAGAIAAAAEDAAAAAAAAAA==/","_ts":1608932852}],"_count":3}
And here the response I get from azure
{"partitionedQueryExecutionInfoVersion":2,"queryInfo":{"distinctType":"None","top":null,"offset":null,"limit":null,"orderBy":[],"orderByExpressions":[],"groupByExpressions":[],"groupByAliases":[],"aggregates":[],"groupByAliasToAggregateType":{},"rewrittenQuery":"","hasSelectValue":true},"queryRanges":[{"min":"05C1A3C5D33F20083200","max":"05C1A3C5D33F20083200","isMinInclusive":true,"isMaxInclusive":true},{"min":"05C1D12BA14B20083300","max":"05C1D12BA14B20083300","isMinInclusive":true,"isMaxInclusive":true},{"min":"05C1D3D3BB9304083400","max":"05C1D3D3BB9304083400","isMinInclusive":true,"isMaxInclusive":true}]}
Note: I had to generate my own CA then generate a certificate with CN=localhost then add the CA to /usr/local/share/ca-certificates/
and run sudo update-ca-certificates
to pass the https check
using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using System.Threading.Tasks;
using Microsoft.Azure.Cosmos;
using Microsoft.Azure.Cosmos.Fluent;
using Microsoft.Azure.Cosmos.Linq;
namespace ReproBugCosmosDb
{
class Program
{
public class SomeObject
{
public string Id { get; set; }
public IList<string> SomeData { get; set; } = new List<string>();
}
private static CosmosClient _cosmosClient;
public const string DevConnectionString = @"AccountEndpoint=https://localhost:8081/;AccountKey=C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==";
public const string DatabaseName = "SomeReproDatabase2";
public const string ContainerId = "SomeContainerName";
static async Task Main(string[] args)
{
_cosmosClient = new CosmosClientBuilder(DevConnectionString)
.WithSerializerOptions(new CosmosSerializationOptions {PropertyNamingPolicy = CosmosPropertyNamingPolicy.CamelCase})
.WithConnectionModeGateway()
.Build();
await ResetDatabaseAsync(DatabaseName);
var database = _cosmosClient.GetDatabase(DatabaseName);
await database.CreateContainerAsync(ContainerId, "/id");
var container = _cosmosClient.GetContainer(DatabaseName, ContainerId);
await container.UpsertItemAsync(new SomeObject {Id = "1", SomeData = new List<string> {"a"}}, new PartitionKey("1"));
await container.UpsertItemAsync(new SomeObject {Id = "2", SomeData = new List<string> {"b"}}, new PartitionKey("2"));
await container.UpsertItemAsync(new SomeObject {Id = "3", SomeData = new List<string> {"c"}}, new PartitionKey("3"));
await container.UpsertItemAsync(new SomeObject {Id = "4", SomeData = new List<string> {"d"}}, new PartitionKey("4"));
try
{
var result = await GetElementsByIds(new List<string> {"1", "2", "3"});
if (!result.OrderBy(x => x).SequenceEqual(new[] {"a", "b", "c"}))
throw new Exception("Invalid result");
Console.WriteLine("Success");
}
catch (CosmosException ex)
{
Console.WriteLine(ex.Diagnostics.ToString());
throw;
}
}
private static async Task ResetDatabaseAsync(string databaseName)
{
await _cosmosClient.CreateDatabaseIfNotExistsAsync(databaseName);
var database = _cosmosClient.GetDatabase(databaseName);
await database.DeleteAsync();
await _cosmosClient.CreateDatabaseIfNotExistsAsync(databaseName);
}
private static async Task<List<string>> GetElementsByIds(IList<string> ids)
{
var container = _cosmosClient.GetContainer(DatabaseName, ContainerId);
var iterator = container.GetItemLinqQueryable<SomeObject>()
.Where(x => ids.Contains(x.Id))
.ToFeedIterator();
var fetchedPhotoDocuments = new List<string>();
while (iterator.HasMoreResults)
{
var item = await iterator.ReadNextAsync().ConfigureAwait(false);
fetchedPhotoDocuments.AddRange(item.Resource.SelectMany(x => x.SomeData).ToList());
}
return fetchedPhotoDocuments.Distinct().ToList();
}
}
}
It throws here https://github.com/Azure/azure-cosmos-dotnet-v3/blob/master/Microsoft.Azure.Cosmos/src/Query/v3Query/CosmosQueryClientCore.cs#L255
And data are loaded here
https://github.com/Azure/azure-cosmos-dotnet-v3/blob/master/Microsoft.Azure.Cosmos/src/Query/Core/Pipeline/CosmosQueryExecutionContextFactory.cs#L217
Exception
Unhandled exception. Microsoft.Azure.Cosmos.CosmosException : Response status code does not indicate success: InternalServerError (500); Substatus: 0; ActivityId: 00000000-0000-0000-0000-000000000000; Reason: (Value cannot be null. (Parameter 'providedRanges'));
---> System.ArgumentNullException: Value cannot be null. (Parameter 'providedRanges')
at Microsoft.Azure.Cosmos.CosmosQueryClientCore.GetTargetPartitionKeyRangesAsync(String resourceLink, String collectionResourceId, List`1 providedRanges, Boolean forceRefresh)
at Microsoft.Azure.Cosmos.Query.Core.ExecutionContext.CosmosQueryExecutionContextFactory.GetTargetPartitionKeyRangesAsync(CosmosQueryClient queryClient, String resourceLink, PartitionedQueryExecutionInfo partitionedQueryExecutionInfo, ContainerQueryProperties containerQueryProperties, IReadOnlyDictionary`2 properties, FeedRangeInternal feedRangeInternal)
at Microsoft.Azure.Cosmos.Query.Core.ExecutionContext.CosmosQueryExecutionContextFactory.TryCreateFromPartitionedQuerExecutionInfoAsync(DocumentContainer documentContainer, PartitionedQueryExecutionInfo partitionedQueryExecutionInfo, ContainerQueryProperties containerQueryProperties, CosmosQueryContext cosmosQueryContext, InputParameters inputParameters, CancellationToken cancellationToken)
at Microsoft.Azure.Cosmos.Query.Core.ExecutionContext.CosmosQueryExecutionContextFactory.TryCreateCoreContextAsync(DocumentContainer documentContainer, CosmosQueryContext cosmosQueryContext, InputParameters inputParameters, CancellationToken cancellationToken)
at Microsoft.Azure.Cosmos.Query.Core.AsyncLazy`1.GetValueAsync(CancellationToken cancellationToken)
at Microsoft.Azure.Cosmos.Query.Core.Pipeline.LazyQueryPipelineStage.MoveNextAsync()
at Microsoft.Azure.Cosmos.Query.Core.Pipeline.NameCacheStaleRetryQueryPipelineStage.MoveNextAsync()
at Microsoft.Azure.Cosmos.Query.Core.Pipeline.CatchAllQueryPipelineStage.MoveNextAsync()
--- End of inner exception stack trace ---
at Microsoft.Azure.Cosmos.CosmosQueryClientCore.GetTargetPartitionKeyRangesAsync(String resourceLink, String collectionResourceId, List`1 providedRanges, Boolean forceRefresh)
at Microsoft.Azure.Cosmos.Query.Core.ExecutionContext.CosmosQueryExecutionContextFactory.GetTargetPartitionKeyRangesAsync(CosmosQueryClient queryClient, String resourceLink, PartitionedQueryExecutionInfo partitionedQueryExecutionInfo, ContainerQueryProperties containerQueryProperties, IReadOnlyDictionary`2 properties, FeedRangeInternal feedRangeInternal)
at Microsoft.Azure.Cosmos.Query.Core.ExecutionContext.CosmosQueryExecutionContextFactory.TryCreateFromPartitionedQuerExecutionInfoAsync(DocumentContainer documentContainer, PartitionedQueryExecutionInfo partitionedQueryExecutionInfo, ContainerQueryProperties containerQueryProperties, CosmosQueryContext cosmosQueryContext, InputParameters inputParameters, CancellationToken cancellationToken)
at Microsoft.Azure.Cosmos.Query.Core.ExecutionContext.CosmosQueryExecutionContextFactory.TryCreateCoreContextAsync(DocumentContainer documentContainer, CosmosQueryContext cosmosQueryContext, InputParameters inputParameters, CancellationToken cancellationToken)
at Microsoft.Azure.Cosmos.Query.Core.AsyncLazy`1.GetValueAsync(CancellationToken cancellationToken)
at Microsoft.Azure.Cosmos.Query.Core.Pipeline.LazyQueryPipelineStage.MoveNextAsync()
at Microsoft.Azure.Cosmos.Query.Core.Pipeline.NameCacheStaleRetryQueryPipelineStage.MoveNextAsync()
at Microsoft.Azure.Cosmos.Query.Core.Pipeline.CatchAllQueryPipelineStage.MoveNextAsync()
--- Cosmos Diagnostics ---{"DiagnosticVersion":"2","Summary":{"StartUtc":"2020-12-25T21:59:18.9399687Z","RunningElapsedTimeInMs":28.6454,"UserAgent":"cosmos-netstandard-sdk/3.15.1|3.15.2|03|X64|Linux 5.4.0-58-generic 64-Ubu|.NET 5.0.1-servicing.20575.16|","TotalRequestCount":0,"FailedRequestCount":0,"Operation":"CosmosDiagnosticsContextCore"},"Context":[]}
Looks like an internal error on the following query (went sent raw, untransformed):
SELECT
COUNT(1)
FROM (
SELECT
c.name
FROM c
WHERE
c.name = 'READY'
) AS c
The read-meta handler hardcodes port 3000, which is incorrect when running the server on another port. This causes the Python client to throw a connection refused exception after connecting to a server on another port:
from azure.cosmos import CosmosClient
cosmos = CosmosClient("https://localhost:32797", credential="fake")
list(cosmos.list_databases())
This throws the exception:
<urllib3.connection.HTTPSConnection object at 0x7fd48863dfd0>: Failed to establish a new connection: [Errno 111] Connection refused
due to the client trying to communicate with port 3000, because of the hard coded values.
This can be solved by passing server options to handlers, so that the port can be dynamically generated; this would be a lot of code changes so perhaps there's an easier way to avoid this?
Combine https://github.com/zeit/cosmosdb-query with this repository
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.