GithubHelp home page GithubHelp logo

nodkz / mongodb-memory-server Goto Github PK

View Code? Open in Web Editor NEW
2.5K 19.0 180.0 16.05 MB

Spinning up mongod in memory for fast tests. If you run tests in parallel this lib helps to spin up dedicated mongodb servers for every test file in MacOS, *nix, Windows or CI environments (in most cases with zero-config).

Home Page: https://nodkz.github.io/mongodb-memory-server/

License: MIT License

JavaScript 7.90% TypeScript 91.75% Shell 0.03% CSS 0.32%
mongodb mongoose mock testing testing-tools tests

mongodb-memory-server's Introduction

MongoDB In-Memory Server

Node.js CI NPM version Downloads stat Commitizen friendly TypeScript compatible codecov.io Backers on Open Collective Sponsors on Open Collective mongodb-memory-server-core

This package spins up an actual/real MongoDB server programmatically from within nodejs, for testing or mocking during development. By default it holds the data in memory. A fresh spun up mongod process takes about 7Mb of memory. The server will allow you to connect your favorite ODM or client library to the MongoDB server and run integration tests isolated from each other.

On install, this package downloads the latest MongoDB binaries and saves them to a cache folder. (only mongodb-memory-server-core does not download on postinstall)

On starting a new instance of the memory server, if the binary cannot be found, it will be auto-downloaded (if RUNTIME_DOWNLOAD option is truthy), thus the first run may take some time. All further runs will be fast, because they will use the downloaded binaries.

This package automatically downloads binaries from https://fastdl.mongodb.org/ according to your operating system. You can see all available versions for Linux (Ubuntu, RHEL, Debian, SUSE, Amazon), OSX, and Windows.

If your network is behind a proxy, make sure that it is configured through the HTTPS_PROXY or HTTP_PROXY environment variable.

Every MongoMemoryServer instance creates and starts a fresh MongoDB server on some free port. You may start up several mongod simultaneously. When you terminate your script or call stop(), the MongoDB server(s) will be automatically shutdown.

Works perfectly with any CI runner that can run NodeJS applications.

Table of Contents generated with DocToc

Installation

This tool provides three packages for different purposes:

  • With auto-download mongod binary on npm install (mongodb-memory-server, mongodb-memory-server-global-*)
  • Without auto-download on npm install (mongodb-memory-server-core)

Choose any package, because they are the same. They differ only in the default configuration, which you may override (see section Available options).

Requirements

  • NodeJS: 14.20.1+
  • Typescript: 5.0+ (if used)

And one of those (on Linux):

  • having lsb-core installed (or any that provides the lsb_release command)
  • having an /etc/os-release file that is compliant to the OS-Release Spec
  • having an /etc/*-release file that is compliant to the OS-Release Spec (and does not include lsb)
  • manually specify which version & system should be used

On Linux, you will also need libcurl4 (or libcurl3 on some older distro versions). This will probably only be an issue on "slim" Docker images.

Choose the Correct Package

Choose the right package for the task

Configuring which mongod binary to use

The default behavior is that version 6.0.14 for your OS will be downloaded. By setting Environment variables you are able to specify which version and binary will be downloaded:

export MONGOMS_DOWNLOAD_URL=https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu1804-4.2.8.tgz
export MONGOMS_VERSION=4.2.8

Usage

Simple server start

A Normal Server can be easily started with:

import { MongoMemoryServer } from 'mongodb-memory-server';

// This will create an new instance of "MongoMemoryServer" and automatically start it
const mongod = await MongoMemoryServer.create();

const uri = mongod.getUri();

// The Server can be stopped again with
await mongod.stop();

Available options for MongoMemoryServer

All options with ? are optional, some options upon specified are required.

const mongod = new MongoMemoryServer({
  instance?: {
    port?: number, // by default choose any free port
    ip?: string, // by default '127.0.0.1', for binding to all IP addresses set it to `::,0.0.0.0`,
    dbName?: string, // by default '' (empty string)
    dbPath?: string, // by default create in temp directory
    storageEngine?: string, // by default `ephemeralForTest`(unless mongodb 7.0.0, where its `wiredTiger`), available engines: [ 'ephemeralForTest', 'wiredTiger' ]
    replSet?: string, // by default no replica set, replica set name
    args?: string[], // by default no additional arguments, any additional command line arguments for `mongod` `mongod` (ex. ['--notablescan'])
    auth?: boolean, // add "--auth" argument, dont use this directly use top-level "auth"
  },
  binary?: {
    version?: string, // by default '6.0.14'
    downloadDir?: string, // see the documentation on what is chosen by default https://nodkz.github.io/mongodb-memory-server/docs/api/config-options#download_dir
    platform?: string, // by default os.platform()
    arch?: string, // by default os.arch()
    checkMD5?: boolean, // by default false OR process.env.MONGOMS_MD5_CHECK
    systemBinary?: string, // by default undefined or process.env.MONGOMS_SYSTEM_BINARY
  },
  // using "auth" will manage "instance.auth"
  auth?: {
    // enable needs to be set to "true", otherwise automatic user creation is by default disabled
    enable?: boolean, // enable automatic user creation
    customRootName?: string, // by default "mongodb-memory-server-root"
    customRootPwd?: string, // by default "rootuser"
    force?: boolean, // force creation of users
    keyfileContent?: string, // by default "0123456789" (only useful for replsets)
    extraUsers?: [{
      // see mongodb documentation https://docs.mongodb.com/manual/reference/method/db.createUser/#definition)
      createUser: string, // user name
      pwd: string, // user password
      roles: UserRoles[], // user roles
      database?: string, // which database the user is created on
      customData?: Record<string, any>, // any arbitrary information, see mongodb documentation
      mechanisms?: ('SCRAM-SHA-1' | 'SCRAM-SHA-256')[],
      authenticationRestrictions?: {
        clientSource?: string;
        serverAddress?: string;
      }[],
      digestPassword?: boolean,
    }],
  },
});

Replica Set start

A ReplicaSet can be easily started with:

import { MongoMemoryReplSet } from 'mongodb-memory-server';

// This will create an new instance of "MongoMemoryReplSet" and automatically start all Servers
const replset = await MongoMemoryReplSet.create({ replSet: { count: 4 } }); // This will create an ReplSet with 4 members

const uri = replset.getUri();

// The ReplSet can be stopped again with
await replset.stop();

Available options for MongoMemoryReplSet

All options are optional.

const replSet = new MongoMemoryReplSet({
  binary: binaryOpts, // same as for MongoMemoryServer
  instanceOpts: [
    {
      args, // any additional instance specific args
      port, // port number for the instance
      dbPath, // path to database files for this instance
      storageEngine, // same storage engine options
    },
    // each entry will result in a MongoMemoryServer (replSet.count will not limit the amount spawned by "instanceOpts")
  ],
  // unless otherwise noted below these values will be in common with all instances spawned:
  replSet: {
    name, // replica set name (default: 'testset')
    auth?: boolean | AutomaticAuth, // enable auth, for options see #available-options-for-mongomemoryserver
    args, // any args specified here will be combined with any per instance args from `instanceOpts`
    count, // number of additional `mongod` processes to start (will not start any extra if instanceOpts.length > replSet.count); (default: 1)
    dbName, // default database for db URI strings
    ip, // by default '127.0.0.1', for binding to all IP addresses set it to `::,0.0.0.0`
    spawn, // spawn options when creating the child processes
    storageEngine, // default storage engine for instance. (Can be overridden per instance)
    configSettings: {
      // Optional settings for 'replSetInitiate' command. See https://docs.mongodb.com/manual/reference/command/replSetInitiate/
      chainingAllowed: boolean, // When true it allows secondary members to replicate from other secondary members. When false, secondaries can replicate only from the primary.
      heartbeatTimeoutSecs: number, // Number of seconds that the replica set members wait for a successful heartbeat from each other. If a member does not respond in time, other members mark the delinquent member as inaccessible.
      heartbeatIntervalMillis: number, // The frequency in milliseconds of the heartbeats.
      electionTimeoutMillis: number, // The time limit in milliseconds for detecting when a replica setโ€™s primary is unreachable.
      catchUpTimeoutMillis: number, // Time limit for a newly elected primary to sync (catch up) with the other replica set members that may have more recent writes.
    },
  },
});

Config Options

Documentation of Config Options

Simple test with MongoClient in Jest

A example test file for a single MongoMemoryServer within jest.

A example test file for multiple MongoMemoryServer within jest.

A example test file for a single MongoMemoryReplSet within jest.

Provide connection string to mongoose

// assuming [email protected]
import mongoose from 'mongoose';
import { MongoMemoryServer } from 'mongodb-memory-server';

const mongoServer = await MongoMemoryServer.create();

(async () => {
  await mongoose.connect(mongoServer.getUri(), { dbName: "verifyMASTER" });

  // your code here
  
  await mongoose.disconnect();
})();

Test Runner Examples

Documentation for Test Runner Integration Examples

Docker Alpine

There isn't currently an official MongoDB release for alpine linux. This means that we can't pull binaries for Alpine (or any other platform that isn't officially supported by MongoDB), but you can use a Docker image that already has mongod built in and then set the MONGOMS_SYSTEM_BINARY variable to point at that binary. This should allow you to use mongodb-memory-server on any system on which you can install mongod manually.

Enable Debug Mode

The Debug mode can be enabled with an Environment-Variable or in the package.json "config" section:

MONGOMS_DEBUG=1 # also available case-insensitive values: "on" "yes" "true"

or

{
  "config": {
    "mongodbMemoryServer": {
      "debug": "1", // also available case-insensitive values: "on" "yes" "true"
    }
  }
}

Also see the Enable Debug Mode Guide.

Contributing

Contributing Guidelines are setup in CONTRIBUTING

Join Our Discord Server

To ask questions or just talk with us, join our Discord Server.

Documentation

Credits

Inspired by

Inspired by alternative runners for mongodb-prebuilt:

Maintainers

Funding

Contributors

This project exists thanks to all the people who contribute.

Backers

Thank you to all our backers! ๐Ÿ™ [Become a backer]

Sponsors

Support this project by becoming a sponsor. Your logo will show up here with a link to your website. [Become a sponsor]

License

MIT

mongodb-memory-server's People

Contributors

ajrdev avatar arthursimas1 avatar as-ajitsingh avatar chronicusua avatar czr avatar dependabot[bot] avatar fracasula avatar greenkeeper[bot] avatar greenkeeperio-bot avatar guillaumervls avatar hasezoey avatar jardakotesovec avatar jloehel avatar jloveridge avatar kolahzary avatar mathieug avatar mhassan1 avatar nathannaveen avatar nhuanhoangduc avatar nodkz avatar robin850 avatar rodrigotopan avatar ronjouch avatar sebaplaza avatar semantic-release-bot avatar skrtheboss avatar sktt avatar spodjasek avatar traviscibot avatar yozhikm avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

mongodb-memory-server's Issues

storageEngine opt of replSet is ignored

storageEngine option on replSet seems to be ignored:

new MongoMemoryReplSet({
  replSet: {
    storageEngine: 'wiredTiger'
  }
})

It falls back to the default memory storage.

Flow error in optional chaining?

That's what I get when running flow against my project:

yarn flow       
yarn run v1.10.1
$ flow
Error: node_modules/mongodb-memory-server/lib/util/MongoBinary.js.flow:27
 27:         process.env?.MONGOMS_DOWNLOAD_DIR || path.resolve(os.homedir(), '.mongodb-binaries'),
                         ^ Unexpected token .

Perhaps I can add node_modules/mongodb-memory-server to flow's ignore file but still.

project tests are failing

@nodkz I was tryign to use your memory-server in my own project. Awesome idea. But I am running into a problem that may have something to do with your project tests you have set up. I attaching npm's log plus a copy of my own run log. Looks like maybe something needs an update. It is failing for a .toMatch method.
2018-06-24T20_08_22_039Z-debug.log
npmRunTest_consoleSnapshot.txt

I my own project, I chose to not use the es6 'import' and just stick with the older 'require' in my conn.js file within test folder. But that should not be causing any problems, should it? node is complaining of my constructor MongodbMemoryServer statement is not a constructor.
conn.txt

slower than actually running on database

So, I have the following code:

const mongoose = require('mongoose');
const { MongoMemoryServer } = require('mongodb-memory-server');
const mongoServer = new MongoMemoryServer();
mongoServer.getConnectionString().then((mongoUri) => {
  mongoose.connect(mongoUri);
});

const { Schema } = mongoose;
const userSchema = new Schema({ name: String });

const User = mongoose.model('User', userSchema);

and then I export the User model, and run my tests.

The surprising thing is that the tests run in 11 seconds or so when I run them on MongoDB normally. (5 seconds waiting on connection + 6 actually running).

While this takes 15 seconds (9 seconds to establish a connection + 6 actually running).

Am I doing something wrong?

I even tried with a more intensive test that queries a database collection a 1000 times, it takes 31 seconds to run on memory, and 28 seconds to run from disk. Is it possible that having the database hosted on an SSD make the difference negligible? AFAIK RAM should be at least 10 times faster than SSD.

Does it work with mongodb 4?

Latest working version of mongo I tried is 3.6.3
When I try latest or 4.0.0/1/2/3 - it downloads binaries and then silently exits the process

(K)Ubuntu 18.04
Node v10.7.0
mongodb-memory-server 2.4.3

Add download hash checking

Without this if the binary is corrupted there's never an error shown to the user.

I unfortunately found this out after using ctrl+c at the start of my tests 4 hours ago while it was still downloading.

ENOTFOUND when trying to connect

After coping the code from the Readme, when trying to connect I get this error -

failed to connect to server [localhost:52141] on first connect [MongoNetworkError: getaddrinfo ENOTFOUND localhost localhost:52141]

The mongoUri returned from getConnectionString is
"mongodb://localhost:52141/62448dc5-37f7-4158-a47e-e8e3b3b4a2c9"

I'm using mongoose 5.2.9
What am I missing ?
Thanks

Auto-download of binaries on install

Hi and thanks for this module,

I use the downloadDir options in my code, so my home folder stays clean of .mongodb-binaries. However the postinstall script is not aware of that and downloads in my home folder. Shouldn't the downloading of binaries after npm install be optional ? (I mean opt-in instead of opt-out)

Am I missing something ?

Cheers

Win10 x64 - Download only extracts mongod.exe

Hi,

On Win10 x64, it's only extracting mongod.exe and throwing various dll errors related to open ssl.
If I manually download and extract all files from the archive link given in the console, it works fine.

The Jest example in README does not work

I tried out the Jest example in the README and it does not work. When I ran some test, there are some error message saying "failed to connect on first connect".

The problem is at this line

mongoose.connect(mongoUri, opts, (err) => {
    if (err) console.error(err);
  });

Since mongoose.connect() returns a promise, we need to call it with await such that the call only returns when the connection is done.

So it should be something like

await mongoose.connect(mongoUri, opts, (err) => {
    if (err) console.error(err);
  });

How I can change binary.downloadDir through env?

const {
downloadDir = path.resolve(os.homedir(), '.mongodb-binaries'),
platform = os.platform(),
arch = os.arch(),
version = 'latest',
} = opts;

Please define this options as follow

const {
  downloadDir = process.env.MONGOMS_DOWNLOAD || path.resolve(os.homedir(), '.mongodb-binaries'),
  platform = process.env.MONGOMS_PLATFORM || os.platform(),
  arch = process.env.MONGOMS_ARCH || os.arch(),
  version = process.env.MONGOMS_VERSION || 'latest',
} = opts;

otherwise how I can change it options on npm install from CI configurations for Gitlab? It can't cache files from project outside, for caching /root/.mongodb-binaries folder. Now I can only change my test files as

before(async () => {
  server = new MongoMemoryServer({ binary: { downloadDir: './.mongodb-binaries' } });
});

But this test will fail by timeout cause start download binaries. And this does not working because on postinstall script binaries anyway will downloads to os.homedir() + '/.monodb-binaries', than on test start binaries will downloads again to /path/to/project/.mongodb-binaries. Than Gitlab does not cache this binaries because job is fail.

Download url format may be broken

Looks like the download url format may have changed and would need to be updated.

Recently when running tests I have been seeing the download fail for mongo binary, indeed if i follow the link (in my case 3.6.0) logged for the download I am seeing a standard s3 empty page like:

image

Log output is as follows using this package with { debug: true }:

Mongo[58989] Starting MongoDB instance with following options: {"port":58989,"dbName":"01d7602e-c235-4d6b-9009-db9f21c189f0","uri":"mongodb://localhost:58989/01d7602e-c235-4d6b-9009-db9f21c189f0","storageEngine":"ephemeralForTest","dbPath":"/var/folders/q2/5tmxbvwj6gs33d926p9_m45h0000gn/T/mongo-mem-51484gWf8ctpeqK84"} +0ms
  console.warn node_modules/node-uuid/uuid.js:48
    [SECURITY] node-uuid: crypto not usable, falling back to insecure Math.random()

  console.log node_modules/mongodb-memory-server/lib/MongoMemoryServer.js:99
    Autostarting MongoDB instance...

  console.log node_modules/mongodb-memory-server/lib/util/MongoBinaryDownload.js:172
    Downloading MongoDB: https://downloads.mongodb.org/osx/mongodb-osx-x86_64-3.6.0.tgz

  Mongo[58989] renamed ~/.mongodb-binaries/mongodb-osx-x86_64-3.6.0.tgz.downloading to ~/.mongodb-binaries/mongodb-osx-x86_64-3.6.0.tgz +8s
  Mongo[59099] Starting MongoDB instance with following options: {"port":59099,"dbName":"e10683f2-87ed-4816-9320-2843d1c6fe35","uri":"mongodb://localhost:59099/e10683f2-87ed-4816-9320-2843d1c6fe35","storageEngine":"ephemeralForTest","dbPath":"/var/folders/q2/5tmxbvwj6gs33d926p9_m45h0000gn/T/mongo-mem-514841O3FXUa6dBXX"} +0ms
  console.log node_modules/mongodb-memory-server/lib/MongoMemoryServer.js:99
    Autostarting MongoDB instance...

  console.error node_modules/jest-jasmine2/build/jasmine/Env.js:198
    Unhandled error

  console.error node_modules/jest-jasmine2/build/jasmine/Env.js:199
    StatusCodeError: 403 - "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>C341FADDCC53868E</RequestId><HostId>rOpZu2a1arUiUY5MRo2eFY3ZGLNWps9d6aYWhEtvTWYUUpJQLO3rdPRKc3s3Ny3jXoEUQA40+9Y=</HostId></Error>"
        at new StatusCodeError (~/myProject/node_modules/request-promise-core/lib/errors.js:32:15)
        at Request.plumbing.callback (~/myProject/node_modules/request-promise-core/lib/plumbing.js:104:33)
        at Request.RP$callback [as _callback] (~/myProject/node_modules/request-promise-core/lib/plumbing.js:46:31)
        at Request.self.callback (~/myProject/node_modules/request/request.js:186:22)
        at Request.emit (events.js:125:13)
        at Request.emit (domain.js:421:20)
        at Request.<anonymous> (~/myProject/node_modules/request/request.js:1163:10)
        at Request.emit (events.js:125:13)
        at Request.emit (domain.js:421:20)
        at IncomingMessage.<anonymous> (~/myProject/node_modules/request/request.js:1085:12)
        at Object.onceWrapper (events.js:217:13)
        at IncomingMessage.emit (events.js:130:15)
        at IncomingMessage.emit (domain.js:421:20)
        at endReadableNT (_stream_readable.js:1101:12)
        at process._tickCallback (internal/process/next_tick.js:152:19)

Feature request: Allow specification of root download URL

Our company's build servers do not have direct access to the internet. Instead, they request packages from an internal Artifactory that we host. Currently, MongoBinaryDownloadUrl hardcodes the download for mongodb binaries to fastdl.mongodb.org.

It would be very nice if we could override that with an environment variable to allow us to point this package at a mirror of the mongodb binaries that we could host. Other packages have a mechanism to do this (e.g. sass_binary_site, chromedriver_cdnurl).

Error when running tests in Docker / CI

Things work great when I'm running on my local machine, but I'm seeing problems getting things to run in my CI pipeline, which is running tests in a Docker containers.

The symptom is that my global.__MONGOD__ appears to be undefined by the time my mongo-environment.js is loaded by the test.

TypeError: Cannot read property 'getConnectionString' of undefined

Here's what my setup.js and my mongo-environment.js files look like:

// setup.js
const MongodbMemoryServer = require('mongodb-memory-server');

const MONGO_DB_NAME = 'jest';

const mongod = new MongodbMemoryServer.default({
  instance: {
    dbName: MONGO_DB_NAME,
  },
  binary: {
    version: '3.4.6',
  },
});

module.exports = () => {
  global.__MONGOD__ = mongod;
  global.__MONGO_DB_NAME__ = MONGO_DB_NAME;
};
// mongo-environment.js
const NodeEnvironment = require('jest-environment-node');

class MongoEnvironment extends NodeEnvironment {
  async setup() {
    // ๐Ÿ‘‡ This line is throwing the error I mentioned above
    this.global.__MONGO_URI__ = await global.__MONGOD__.getConnectionString();
    this.global.__MONGO_DB_NAME__ = global.__MONGO_DB_NAME__;

    await super.setup();
  }

  async teardown() {
    await super.teardown();
  }

  runScript(script) {
    return super.runScript(script);
  }
}

module.exports = MongoEnvironment;

It looks like mongo doesn't get downloaded all the way before the code tries to run and reference the connection string? As I'm watching it run, I see this line about downloading Mongo and it starts showing percentages, but then tests start running and failing. (Note, the percent is gone in the screen grab).

image

[Windows] Module tries to remove temp dir before mongod exits

This causes

Error: EBUSY: resource busy or locked, unlink 'C:\Users\Gerard\AppData\Local\Temp\mongo-mem-13616GqtHuLWrpEBT\mongod.lock'

Solution would be to wait for the childProcess exit event before calling tmpDir.removeCallback()

Happy to provide a PR

MongoDB always runs on default random port

In memory Mongo is instantiated like this:
const mongod = new MongodbMemoryServer.default({ instance: { dbName: 'my-test-db', port: 12345 } })

It's expected to have MongoDB running on port 12345, but that's not the case. Calling getConnectionString and getPort show that MongoDB is actually running on some default random port (e.g. mongodb://localhost:54271/my-test-db).

This started happening on version 1.9.3 and also happens on 2.0.0.

Download path inconsistency between post-install and defaults.

Currently, the post-install script downloads the binaries to /node_modules/mongodb-memory-server/node_modules/.cache/mongodb-memory-server/mongodb-binaries/latest/mongod, which I believe is due to the cwd being set to /node_modules/mongodb-memory-server when post-install is run. At runtime, the binaries are downloaded anew to /node_modules/.cache/mongodb-memory-server/mongodb-binaries. Perhaps the easiest fix for this would be checking both directories in MongoBinary.js ?

To reproduce:

docker run -it --rm node:11 /bin/bash
root@444a1d59d65f:/# npm init --yes
Wrote to /package.json:

{
  "name": "",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "directories": {
    "lib": "lib"
  },
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC"
}


root@444a1d59d65f:/# npm install --save-dev mongodb-memory-server

> [email protected] postinstall /node_modules/mongodb-memory-server
> node ./postinstall.js

mongodb-memory-server: checking MongoDB binaries cache...
mongodb-memory-server: binary path is /node_modules/mongodb-memory-server/node_modules/.cache/mongodb-memory-server/mongodb-binaries/latest/mongod
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN [email protected] requires a peer of mongodb@>= 3.0 but none is installed. You must install peer dependencies yourself.
npm WARN @1.0.0 No description
npm WARN @1.0.0 No repository field.

+ [email protected]
added 81 packages from 48 contributors and audited 166 packages in 35.706s
found 0 vulnerabilities

root@444a1d59d65f:/# node
> const path = require('path');
undefined
> const findCacheDir = require('find-cache-dir');
undefined
> path.resolve(findCacheDir({ name: 'mongodb-memory-server' }), 'mongodb-binaries');
'/node_modules/.cache/mongodb-memory-server/mongodb-binaries'

regeneratorRuntime is not defined

I'm getting the following error when trying to use this package on node 4.4.7.
From what I can see after googling around a bit, this may require babel-polyfill or similar. I'm not familiar with babel otherwise I'd attempt to put up a pull request.

  โ— Test suite failed to run

    ReferenceError: regeneratorRuntime is not defined

      at node_modules/mongodb-memory-server/lib/MongoMemoryServer.js:10:32
      at Object.<anonymous> (node_modules/mongodb-memory-server/lib/MongoMemoryServer.js:44:2)
      at Object.<anonymous> (node_modules/mongodb-memory-server/lib/index.js:8:26)

What about mongodb-download?

There is interesting package: mongodb-download
What do you think about it?
It looks bit unattended, but may be it's still better to use it for mongo downloading?
Or reuse some code from it?

Incorrect md5 on CircleCI

NPM install is crashing due to incorrect MD5.

mongodb-memory-server: checking MongoDB binaries cache...
Downloading: https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-debian-latest.tgz
Downloading: https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-debian-latest.tgz.md5
(node:2047) UnhandledPromiseRejectionWarning: Error: MongoBinaryDownload: md5 check is failed
    at MongoBinaryDownload._callee3$ (/home/circleci/app/node_modules/mongodb-memory-server/lib/util/MongoBinaryDownload.js:217:23)
    at tryCatch (/home/circleci/app/node_modules/regenerator-runtime/runtime.js:62:40)
    at Generator.invoke [as _invoke] (/home/circleci/app/node_modules/regenerator-runtime/runtime.js:296:22)
    at Generator.prototype.(anonymous function) [as next] (/home/circleci/app/node_modules/regenerator-runtime/runtime.js:114:21)
    at step (/home/circleci/app/node_modules/mongodb-memory-server/lib/util/MongoBinaryDownload.js:52:191)
    at /home/circleci/app/node_modules/mongodb-memory-server/lib/util/MongoBinaryDownload.js:52:437
    at new Promise (<anonymous>)
    at MongoBinaryDownload.<anonymous> (/home/circleci/app/node_modules/mongodb-memory-server/lib/util/MongoBinaryDownload.js:52:99)
    at MongoBinaryDownload.checkMd5 (/home/circleci/app/node_modules/mongodb-memory-server/lib/util/MongoBinaryDownload.js:228:22)
    at MongoBinaryDownload._callee2$ (/home/circleci/app/node_modules/mongodb-memory-server/lib/util/MongoBinaryDownload.js:179:29)
(node:2047) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 2)

Just a question with migration

Hello, when we are executing our test with the real mongodb engine we use migrate-mongo to execute script to prepare tests database filling required data. Since It seems we are having issue with Jest and MongoDb related ton an openHandle, how could we run our migration on your in memory database?

Thanks

MongoError differ between mongodb-memory-server and vanilla mongod

First off, I totally love mongodb-memory-server - it makes testing models and GraphQL APIs a breeze!

If I log the errmsg prop of a MongoError like this:

try {
  await col.insertOne(doc);
} catch (error) {
  if (error typeof MongoError && error.code === 11000) {
    console.log(error.errmsg);
  }
}

I get this with mongodb-memory-server (Jest):

E11000 duplicate key error dup key: { : "YF2dqu11Z9mj", : "test" }

And this with a vanilla mongod process (Node):

E11000 duplicate key error collection: myDatabase.myCollection index: organizationId_1_name_1 dup key: { : "YF2dqu11Z9mj", : "test" }

I parse errmsg to figure out which index clashed so I can provide a more detailed error message to the client. But since the index part of errmsg is not available in the MongoError from mongodb-memory-server I can't write any tests for it.

Is there any way around this? Could it be that Jest is the reason for this?

MongoDB binaries mismatch checking

in postInstall.js

if (isModuleExists('./lib/util/MongoBinary')) {
  const MongoBinary = require('./lib/util/MongoBinary').default;

  console.log('mongodb-memory-server: checking MongoDB binaries cache...');
  MongoBinary.getPath({
    version: 'latest',
  }).then(binPath => {
    console.log(`mongodb-memory-server: binary path is ${binPath}`);
  });
}

this will be download mongodb latest version to skip timeout setup jasmine.DEFAULT_TIMEOUT_INTERVAL = 60000;

but in /util/MongoBinary.js

export default class MongoBinary {
  static cache: MongoBinaryCache = {};

  static async getPath(opts?: MongoBinaryOpts = {}): Promise<string> {
    const {
      downloadDir = path.resolve(os.homedir(), '.mongodb-binaries'),
      platform = os.platform(),
      arch = os.arch(),
      version = '3.4.4',
} = opts;

the mongodb default version is 3.4.4

Therefore, it will always download mongodb-binaries version 3.4.4 if not define a version on the first run

If we make this 2 points (mongodb version) to the same version then this problem will solved?

[Windows] Incorrect `mongod` binary path

Freshly installed on a new machine
Windows10 x64

Autostarting MongoDB instance...
  Mongo[50146] Starting MongoDB instance with following options: {"port":50146,"dbName":"f63e96d0-2244-497d-a043-f608688a503d","uri":"mongodb://localhost:50146/f63e96d0-2244-497d-a043-f608688a503d","storageEngine":"ephemeralForTest","dbPath":"C:\\Users\\Username\\AppData\\Local\\Temp\\mongo-mem-6080BA3QCuF1h8UT"} +0ms
Downloading MongoDB: https://downloads.mongodb.org/win32/mongodb-win32-x86_64-3.4.4.zip
  Mongo[50146] renamed C:\Users\Username\.mongodb-binaries\mongodb-win32-x86_64-3.4.4.zip.downloading to C:\Users\Username\.mongodb-binaries\mongodb-win32-x86_64-3.4.4.zip +10s
  Mongo[50146] getDownloadMD5Hash content: 024a46524c856429d0fd9de0c7d04c5f  mongodb-win32-x86_64-3.4.4.zip
  Mongo[50146]  +557ms
  Mongo[50146] getDownloadMD5Hash extracted signature: 024a46524c856429d0fd9de0c7d04c5f +1ms
  Mongo[50146] extract(): C:\Users\Username\.mongodb-binaries\3.4.4 +581ms
(node:6080) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): Error: MongoBinaryDownload: missing mongod binary in C:\Users\Username\.mongodb-binaries\mongodb-win32-x86_64-3.4.4.zip
(node:6080) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 2): Error: MongoBinaryDownload: missing mongod binary in C:\Users\Username\.mongodb-binaries\mongodb-win32-x86_64-3.4.4.zip

Unique-Indexes seems not working.

Hi,
i have this schema-index:

profilesSchema.index(
	{
		type: 1,
		name: 1,
	},{
		unique: true,
	}
);

type and name is a combined index. This works live, but in the test i can add two documents with the same type and the same name.

I am using binary: {version: '4.0.3' } on both sides.

Are unique indexes are supported, yet?

Update readme

It worth to remove 'use mongodb-prebuild' clause from readme. No one reads the docs thought =)

Linux download for Ubuntu 18.04 fails

Hi,

I have tried to run mongodb-memory-server ([email protected]) as a fresh npm install on Ubuntu 1804.

It is failing with an incorrect download location. Please see the following stack trace: -

Determining test suites to run... Mongo[40427] Starting MongoDB instance with following options: {"port":40427,"dbName":"jest","uri":"mongodb://127.0.0.1:40427/jest","storageEngine"
:"ephemeralForTest","dbPath":"/tmp/mongo-mem-15254m9v8farYN312"} +0ms
Mongo[40427] MongoBinary options: {"downloadDir":"/home/jenkins/.mongodb-binaries","platform":"linux","arch":"x64","version":"3.2.18"} +2ms
Mongo[40427] Downloading: https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu1804-3.2.18.tgz +27ms
Mongo[40427] renamed /home/jenkins/.mongodb-binaries/mongodb-linux-x86_64-ubuntu1804-3.2.18.tgz.downloading to /home/jenkins/.mongodb-binaries/mongodb-linux-x86_64-ubuntu1804-3.2.1
8.tgz +97ms
Mongo[40427] Downloading: https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu1804-3.2.18.tgz.md5 +1ms
Mongo[40427] renamed /home/jenkins/.mongodb-binaries/mongodb-linux-x86_64-ubuntu1804-3.2.18.tgz.md5.downloading to /home/jenkins/.mongodb-binaries/mongodb-linux-x86_64-ubuntu1804-3
.2.18.tgz.md5 +10ms
Mongo[40427] extract(): /home/jenkins/.mongodb-binaries/3.2.18 +1ms
Error: MongoBinaryDownload: missing mongod binary in /home/jenkins/.mongodb-binaries/mongodb-linux-x86_64-ubuntu1804-3.2.18.tgz
at /var/lib/jenkins/workspace/PrismIS/node_modules/mongodb-memory-server/lib/util/MongoBinaryDownload.js:207:15
at Generator.next ()
at asyncGeneratorStep (/var/lib/jenkins/workspace/PrismIS/node_modules/mongodb-memory-server/lib/util/MongoBinaryDownload.js:28:103)
at _next (/var/lib/jenkins/workspace/PrismIS/node_modules/mongodb-memory-server/lib/util/MongoBinaryDownload.js:30:194)
at

Trying this URL: https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu1804-3.2.18.tgz gives a 'permission denied'.

Kind regards.

Peter

Silently failing in manjaro linux

Hi,

I'm trying to configure jest to use this package but I've found that doesn't work with manjaro. This is the error I get when I activate the debugging:

Determining test suites to run...  Mongo[45603] Starting MongoDB instance with following options: {"port":45603,"dbName":"jest","uri":"mongodb://127.0.0.1:45603/jest","storageEngine":"ephemeralForTest","dbPath":"/tmp/mongo-mem-16705EyirF2mqsJ7s"} +0ms
  Mongo[45603] MongoBinary options: {"downloadDir":"/home/german/Proyectos/dyd/dyd-plus/node_modules/.cache/mongodb-memory-server/mongodb-binaries","platform":"linux","arch":"x64","version":"latest"} +1ms
  Mongo[45603] MongoBinary: Mongod binary path: /home/german/Proyectos/dyd/dyd-plus/node_modules/.cache/mongodb-memory-server/mongodb-binaries/latest/mongod +6ms
  Mongo[45603] MongoBinary: Download lock removed +6ms
  Mongo[45603] STDERR: /home/german/Proyectos/dyd/dyd-plus/node_modules/.cache/mongodb-memory-server/mongodb-binaries/latest/mongod: /usr/lib/libcurl.so.4: version `CURL_OPENSSL_3' not found (required by /home/german/Proyectos/dyd/dyd-plus/node_modules/.cache/mongodb-memory-server/mongodb-binaries/latest/mongod)
  Mongo[45603]  +1ms
  Mongo[45603] CLOSE: 1 +0ms
Done in 2.70s.

Seems like it is downloading the ubuntu's version which does not work with manjaro, I've tried manually downloading the generic one and it works well. Besides that I think whould be very useful to throw some exception in this case. I checked the code and maybe it can be changed around there https://github.com/nodkz/mongodb-memory-server/blob/master/src/util/MongoInstance.js#L148 to handle an exit codes different than 0, but I'm not realy sure.

Greetings.

Feature: fetch mongodb on package install

I've noticed substantial delay in e2e tests due to on-demand mongodb server download.
On the other hand, on-demand download makes testing without internet access almost impossible because we've to mock 3rd-party infrastructure.

It worth to provide a way to have mongodb server as a part of this npm package or add some optional npm dependency (separate package) to take server from.

Probably incorrect documentation

I'm trying to use the module as described in docs.

const MongodbMemoryServer = require('mongodb-memory-server');
const mongoServer = new MongodbMemoryServer();

But I got the error TypeError: MongodbMemoryServer is not a constructor.

At this page I've found another piece of code

const mongoServer = new MongodbMemoryServer.MongoMemoryServer();

and that works. Is that an error in the documentation or in the code?

Doesn't check that binaries have been already downloaded

Hi,

I got an issue that it is downloading binaries twice: after npm install and when instance is going to run.

I have a problem in this case:
In my CI/CD job I run inside docker container npm i and npm test commands and it always download binaries at least 1 time.
Please add checking that binaries have been already downloaded.

The request module is required by request-promise

I see that you've installed request-promise as dependency without its peer request. Now when you install the package, you'll run in the following warnings;

warning "mongodb-memory-server > [email protected]" has unmet peer dependency "request@^2.34".
warning "mongodb-memory-server > request-promise > [email protected]" has unmet peer dependency "request@^2.34".

As Yarn does not install peer dependencies automatically.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.