db-migrate / mongodb Goto Github PK

I get this warning and no migrations run when calling db-migrate up

Here is my setup:

db-migrate version: 1.0.0-beta15
db-migrate-mongodb version: 1.5.0

in my migrations folder I have setup a package.json file with the following:

{
  "type": "module"
}

so that I can use ESM and imports in my migrations.

Hi,
Can you add support for creating Index in background?
https://docs.mongodb.com/v3.0/tutorial/build-indexes-in-the-background/

Hi I am struggling to get this package working. Seems to me that I have connection to mongo db setup properly, because I could list my collections names from calling db._getCollectionNames

but I am not able to figure out how to get collection, so I can perform update on collection:

exports.up = function (db) {
    // how to use db to get to collection?
    // this won't work
    db.collection('Profile').update({},
        { $set: { "termsOfServiceUpdated": true } },
        {
            upsert: false,
            multi: true
        });
};

thank you very much

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

There is a high severity vulnerability in the current version (1.5.0) - Denial of Service
https://npmjs.com/advisories/1203

It does not pass npm audit. Please upgrade mongodb to version >=3.1.13 so it passes.

There is a $20 open bounty on this issue. Add to the bounty at Bountysource.

Hi,
This package is not supporting latest mongoDB version 6^, We are using this package for migration, now we are upgrading our application mongodb version to 6^, this package is creating problem , we are stuck because this package not supporting latest mongodb driver version 6^.

below code needs to be corrected

db = config.db || new MongoClient(new Server(host, port)); in index.js file, because new Server() is not supported in latest mongodb driver.

See db-migrate/node-db-migrate#331 for the issue

Issue description

I am struggling with using db-migrate with MongoDB.

Expected Behavior

I would like to see created index for specific field of my collection on mongodb by using db-migrate

Actual Behavior

[ERROR] AssertionError [ERR_ASSERTION]: ifError got unwanted exception: Unsupported OP_QUERY command: listCollections. The client driver may require an upgrade. For more details see https://dochub.mongodb.org/core/legacy-opcode-removal
at module.exports (/Users/cihataydin/Milvus/microservice-nestjs-template/node_modules/db-migrate/lib/commands/helper/assert.js:9:14)
at /Users/cihataydin/Milvus/microservice-nestjs-template/node_modules/db-migrate/lib/commands/up.js:29:16
at tryCatcher (/Users/cihataydin/Milvus/microservice-nestjs-template/node_modules/bluebird/js/release/util.js:16:23)
at Promise.errorAdapter [as _rejectionHandler0] (/Users/cihataydin/Milvus/microservice-nestjs-template/node_modules/bluebird/js/release/nodeify.js:35:34)
at Promise._settlePromise (/Users/cihataydin/Milvus/microservice-nestjs-template/node_modules/bluebird/js/release/promise.js:601:21)
at Promise._settlePromise0 (/Users/cihataydin/Milvus/microservice-nestjs-template/node_modules/bluebird/js/release/promise.js:649:10)
at Promise._settlePromises (/Users/cihataydin/Milvus/microservice-nestjs-template/node_modules/bluebird/js/release/promise.js:725:18)
at _drainQueueStep (/Users/cihataydin/Milvus/microservice-nestjs-template/node_modules/bluebird/js/release/async.js:93:12)
at _drainQueue (/Users/cihataydin/Milvus/microservice-nestjs-template/node_modules/bluebird/js/release/async.js:86:9)
at Async._drainQueues (/Users/cihataydin/Milvus/microservice-nestjs-template/node_modules/bluebird/js/release/async.js:102:5)
at Async.drainQueues [as _onImmediate] (/Users/cihataydin/Milvus/microservice-nestjs-template/node_modules/bluebird/js/release/async.js:15:14)
at process.processImmediate (node:internal/timers:478:21)
at MongoError.create (/Users/cihataydin/Milvus/microservice-nestjs-template/node_modules/mongodb-core/lib/error.js:31:11)
at queryCallback (/Users/cihataydin/Milvus/microservice-nestjs-template/node_modules/mongodb-core/lib/cursor.js:212:36)
at /Users/cihataydin/Milvus/microservice-nestjs-template/node_modules/mongodb-core/lib/connection/pool.js:469:18
at process.processTicksAndRejections (node:internal/process/task_queues:77:11)

Steps to reproduce

1. Create database.json file

{
  "mongo": {
    "migrationsTableName": "_migrations",
    "driver": "mongodb",
    "host": {"ENV": "DATABASE_HOST"},
    "port": {"ENV": "DATABASE_PORT"},
    "username": {"ENV": "DATABASE_USERNAME"},
    "password": {"ENV": "DATABASE_PASSWORD"},
    "authSource": {"ENV": "DATABASE_AUTH_SOURCE"},
    "authMechanism": "DEFAULT",
    "database": "test",
    "multipleStatements": true,
    "migrationsDir": "migrations",
    "useNewUrlParser": true,
    "useUnifiedTopology": true
  }
}

2. Create migration file
   db-migrate create mongo-index-test --config database.json -e mongo
3. Fill the up and down functions

const { MongoClient } = require("mongodb");
exports.up = async function(db) {
  const client = await MongoClient.connect(db.connectionString);
  const database = client.db('test');

  await database.collection('examples').createIndex({ name: 1 }, { name: 'test_index' });

  await client.close();
};

exports.down = async function(db) {
  const client = await MongoClient.connect(db.connectionString);
  const database = client.db('test');

  await database.collection('examples').dropIndex('test_index');

  await client.close();
};

4. Run migrations
   db-migrate up --config database.json -e mongo

My Environment

Operating System macOS Monterey version 12.5
Node.js version 20.8.0
Typescript version 5.2.2
db-migrate 0.11.14
db-migrate-mongodb 1.5.0
mongodb driver version 6.2.0
docker container with mongodb version 7.0.1

Note: If I use Docker container with MongoDB version 5.0, the error will disappear, and the migration will work well.

I would like to add to the mongo db driver wrapper support for authdb authentication.
I have added this implementation.

I found this error Error: global leak detected: extraParams when I start testing with mocha --check-leaks in my project with using db-migrate.

Can you please fix this?

I can see the version of mongodb. Can we have @3.1.10? I am having some issue with the version.

[email protected]

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Have a provision to pass sparse as an option while creating index.

Hi
It would be great to have a typescript definition file

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

There's no handling for failure of the MongoClient, I purpose we add some.

Ex, if the mongo instance is down, the following is thrown:

[ERROR] TypeError: Cannot read property 'createCollection' of null
    at ./node_modules/db-migrate-mongodb/index.js:363:15
    at connectCallback (./node_modules/db-migrate-mongodb/node_modules/mongodb/lib/mongo_client.js:527:5)
    at ./node_modules/db-migrate-mongodb/node_modules/mongodb/lib/mongo_client.js:418:11
    at _combinedTickCallback (internal/process/next_tick.js:131:7)
    at process._tickCallback (internal/process/next_tick.js:180:9)

The other drivers do this:

https://github.com/db-migrate/mysql/blob/master/index.js#L508-L514

Cheers

the method close for the mondo driver does nothing other then return a successfull promise, which means that the migration process doesn't close the connection

   /**
   * Closes the connection to mongodb
   */
   close: function(callback) {
     return Promise.resolve().nodeify(callback);
   }

From this example:
https://docs.mongodb.com/manual/tutorial/geospatial-tutorial/#prerequisites

It would be nice if db-migrate could support this:
db.restaurants.createIndex({ location: "2dsphere" })

This currently can't be done with addIndex NoSQL API.

Workaround: loading native mongodb driver:

const MongoClient = require('mongodb').MongoClient;
const assert = require('assert');
const url = 'mongodb://mongo:27017/test';

exports.up = function(db, callback) {
  MongoClient.connect(url, function(err, dbnative) {
        assert.equal(null, err);
        assert.notEqual(null, dbnative);
        dbnative.collection('ubicaciones').createIndex({ location: "2dsphere" })
        dbnative.close();
        callback();
      });
  return null;
}

<bountysource-plugin>

---
Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/46557329-add-support-for-mongo-2dsphere-indexes?utm_campaign=plugin&utm_content=tracker%2F12293389&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F12293389&utm_medium=issues&utm_source=github).
</bountysource-plugin>

Functionality described here : https://docs.mongodb.com/manual/core/index-ttl/

Could be combined with issue #13 to support 'background' option when creating indexes as well.

I'm submitting a...

• Bug report
• Feature request
• Question

Current behaviour

exports.down = function(db) {
  return db.dropTable("pets");
};

// Returns:
[INFO] Defaulting to running 1 down migration.
collection migrations
collection pets

The code above remove the collection pets but doesn't remove the document in the migration collection.

exports.down = function(db, callback) {
  db.dropTable("pets1", callback);
};

// Returns:
[INFO] Defaulting to running 1 down migration.
collection migrations
collection pets1
[INFO] Processed migration 20190704145216-yop2
[INFO] Done

The code above works as expected, it removes the collection pets1 and remove the right document in the migration collection.

Expected behaviour

It should remove the right document in the migration collection when it succeed.

Environment

[email protected]
[email protected]
Node version: v12.4.0
Platform:   Mac

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

it seems that there is missed implementation of remove method https://github.com/db-migrate/english-docs/blob/master/API/Seeder.md#removetable-ids-callback

Hi there,

How can I use db-migrate to create a text index on a field like specified in the doc here? https://docs.mongodb.com/manual/core/index-text/#create-text-index

I am struggling with the available methods and as far as I understand, one can only create an index on a field in order to set it as unique, but no more.

If you have any clue I’d be happy.

Hey again @wzrdtales !

Sorry if the title is unclear. Right now in the mongo driver we have the following:

db = config.db || new MongoClient(new Server(host, port));

And I'm hoping to change it to something which has support for ReplSet and associated options:

    db = config.db || new MongoClient(new ReplSet(servers, options));

I think my general approach would be to try to avoid changing the API and add additional optional options to the config. Perhaps something like:

var mongoClient;
if (config.replSet) {
  var hosts = config.replSet.split(',');
  var servers = hosts.map(function(host) {
    return new Server(host, port, config.replSetOptions);
  });
  db = config.db || new MongoClient(new ReplSet(servers, config.replSetOptions));
} else {
  db = config.db || new MongoClient(new Server(host, port));
}

Before I went ahead and figured the rest out I figured I'd double check with you that this seems reasonable and you'd be amenable to a PR for this. Also please let me know if you have any implementation requests. And, as always, thanks for the work you do :)

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

_run function (see below) is not documented (https://db-migrate.readthedocs.io/en/latest/API/programable/#run)

/**

• Gets a connection and runs a mongo command and returns the results
• @param command - The command to run against mongo
• @param collection - The collection to run the command on
• @param options - An object of options to be used based on the command
• @param callback - A callback to return the results
  */
  _run: function(command, collection, options, callback) {

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

It seems I can't use the modern URL format for connecting (eg, 'mongodb://host:port/database') - when I do, it tells me I need to specify the database parameter.

Would anyone object if I did a PR that allowed for both the existing object and the URL based connection string?

Hey @wzrdtales,
This project hasn't gone through a MongoDB driver upgrade in a while. Is this something that you would be welcoming as an upstream contribution?
I would be looking into upgrading the MongoDB driver to 4.x since it includes support for MongoDB AWS IAM Users.

Refers to db-migrate/node-db-migrate#330

Was implemented in bc6fea2

Hello, I'm wondering if migrated scripts log can be stored into mongodb? I mean without using external file to track migrated script.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Looking at https://github.com/db-migrate/mongodb/blob/master/index.js#L337 the deleteMany method is not going to work.

We check if options is an array if(util.isArray(options)) and then add it in the query.

case 'remove':
  // options is the records to insert in this case
  if(util.isArray(options))
    db.collection(collection).deleteMany(options, callbackFunction);
  else
    db.collection(collection).deleteOne(options, callbackFunction);
  break;

The issue is filter from deleteMany(filter) is expected to be an object and not an array and will return an error: MongoError: BSON field 'delete.deletes.q' is the wrong type 'array', expected type 'object'

Here the code I was using:

db._run("remove", "pets3", { id: "003" }, callback); // This removes one entry.
db._run("remove", "pets3", [{ id: "003" }], callback); // This fails obviously.

// This code works and it's using directly the mongo instance.
const getDbInstance = await db._run("getDbInstance");
getDbInstance.collection("pets3").deleteMany({ id: "003" }); // This removes every entries.
getDbInstance.close();

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.