devconcept / multer-gridfs-storage Goto Github PK

Error: The database connection must be open to store files
at GridFSStorage._handleFile

Feature Request

So I'm using MongoDB Atlas, Mongoose, Multer, and GridFsStorage to upload files to a single database on the cloud which works fine, but I want to be able to upload files to different databases, using different database connections dynamically.

In order to do this, I want to set up my Multer GridFsStorage configuration so that the "db" field changes dynamically based on the API request. However, I can't access the HTTP request in the "db" field, which would make it impossible for me to change the DB connection dynamically and it seems to me that it is not currently possible do this.

The solution I'd like to see

I would like to see a way to make the GridFsStorage configuration work dynamically with multiple DB connections.
Ideally, the "db" field in the GridFsStorage configuration would be able to function like the "file" field where the request is accessible through the parameters of a function:

const storage = new GridFsStorage({
    
    // I want to be able to access the request through a function's paramaters. That way I'll be able to return  
    // the database connection back to the "db" field dynamically based on the request's cookies.
    db: (req) => {
          // Determine the DB connection dynamically using the req parameter.
          dynamicConnection = connectionFactory.getDatabaseConnection(req.cookies.dbName);

          // Return the DB connection back to the db field.
          return dynamicConnection;
    }
    
    // The fact that I have access to the user's HTTP request in this method is excellent.
    file: (req, file) => { 
          ...
    }
});


const fileUpload = multer({ 
    // Set the storage strategy.
    storage: storage,

    limits: { fileSize: 1024 * 1024 * 2 },
    fileFilter: fileFilter,
});

I would like to retrieve the ObjectId (or even filename) from the callback, and this even if there was an error. At least, I have not figured out how to do so. To illustrate, this is what I am trying to achieve.

I would like to have efficient, implicit deduplication when a document is inserted in GridFS. My root name is "attachments", for this example. To do so, I have created a unique index on the "md5" field in the attachment.files collection. This means that there should be an error if a filename with the same md5 hash is inserted. And I can then handle it, although I don't know how to retrieve the ObjectID or md5 hash from this transaction.

There is a problem with this: when the document is inserted, It is first created in attachments.chunks, and then hashed, and only then the attachment.files entry is attempted to be created, and raises the error if there's a duplicate. Which means there are dangling chunks with no corresponding files entry. Again, if I could know what the ObjectId was at the moment the attempt was made, I could handle it and simply remove the duplicate chunks. How do I do this?

My workaround is a naive approach where I manually md5 hash the file before and checking if it exists before attempting to insert, but that seems to me inefficient, since it will be hashed a second time.

(node:6411) DeprecationWarning: collection.insert is deprecated. Use insertOne, insertMany or bulkWrite instead. (Mongodb v3)

how to upload images in diskStorage like we use in multer.diskStorage().

when readStream on error,there is not response on storage.fromStream.
i fund it, gridfs.js readstream not handle readStream on error
/_multer-gridfs-storage@4.2.0@multer-gridfs-storage/lib/gridfs.js

async fromStream(readStream, request, file) {
...
  writeStream.on('error', emitError);
}

You might run into issues when connecting using mongoose 5.0 while following the examples provided here and on gridfs_stream. So Consider using mongoose V4.. for gridfs and mongoose 5.0 for your mongoDB .on how to install same packages of different version follow this https://gist.github.com/mnpenner/c46a5eff34cfa91f361a8b3baa249a10

Describe the bug
When used with mongodb 3.6.4 will show depracation waring

(node:10148) DeprecationWarning: Listening to events on the Db class has been deprecated and will be removed in the next major version.

Mongodb wan't us to use the client object for events and not the DB object
Environment
multer-gridfs-storage: 4.2.0
mongodb: 3.6.4

Expected behavior
Not showing the warning.
I created a simple PR to fix it

I am having an error in starting my app.

see below:

{ MongoError: Authentication failed.
at /var/www/app-api/node_modules/mongodb-core/lib/connection/pool.js:581:63
at authenticateStragglers (/var/www/app-api/node_modules/mongodb-core/lib/connection/pool.js:504:16)
at Connection.messageHandler (/var/www/app-api/node_modules/mongodb-core/lib/connection/pool.js:540:5)
at emitMessageHandler (/var/www/app-api/node_modules/mongodb-core/lib/connection/connection.js:310:10)
at Socket. (/var/www/app-api/node_modules/mongodb-core/lib/connection/connection.js:453:17)
at emitOne (events.js:116:13)
at Socket.emit (events.js:211:7)
at addChunk (_stream_readable.js:263:12)
at readableAddChunk (_stream_readable.js:250:11)
at Socket.Readable.push (_stream_readable.js:208:10)
at TCP.onread (net.js:601:20)
ok: 0,
errmsg: 'Authentication failed.',
code: 18,
codeName: 'AuthenticationFailed',
name: 'MongoError',
[Symbol(mongoErrorContextSymbol)]: {} }
(node:20192) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'db' of undefined
at Object.getDatabase (/var/www/app-api/node_modules/multer-gridfs-storage/lib/utils.js:187:11)
at GridFSStorage._setDb (/var/www/app-api/node_modules/multer-gridfs-storage/lib/gridfs.js:238:19)
at promise.then (/var/www/app-api/node_modules/multer-gridfs-storage/lib/gridfs.js:173:14)
at
at process._tickCallback (internal/process/next_tick.js:189:7)
(node:20192) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:20192) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

i am using

multer-gridfs-storage: ^3.2.3
mongodb: 4.0.4
multer: ^1.4.1
gridfs-stream: ^1.1.1
mongoose: ^5.3.4

In my laptop i don't have this kind of problem. but after i upload it to ubuntu 18.0 virtual machine. the error occurred.

Please i need help.

Thank you in advance!

how set custom names for fields like filename or set metadatas can you please give an example ?

Describe the bug

Using the mongoose connection as a db throws this warning:

Listening to events on the Db class has been deprecated and will be removed in the next major version.

My code looks like this:

const { connection } = require('mongoose');
const { GridFsStorage } = require('multer-gridfs-storage');

const storage = new GridFsStorage({
  db: connection,
});

Environment

• I'm using multer-gridfs-storage version 5.0.1

• I have Multer 1.4.3 installed to upload files

• The Node version used to run the code is 16.3.0

• I'm using Mongoose connection objects to create storage instances. The Mongoose version installed is 5.12.13

After upgrading to the latest version of mongoose, this line throws an exception.

Describe the bug

In the file handler, I can't see the file.webkitRelativePath passed by the frontend client.
I would like to preserve the relative path when uploading a folder.

Also, file.originalName is only left with the filename, the relativepath is gone.

            let fileHandler = (req: any, file: any) => { 
                debuglog( () => { return JSON.stringify(file)})
                return { bucketName: type, metadata: { id: req.body.id as string, username: req.user.username as string}, filename: file.webkitRelativePath? file.webkitRelativePath: file.originalName }
            }
            storage = new GridFsStorage({
                db: db,
                cache: true,
                file: fileHandler
            });

Environment

• I'm using multer-gridfs-storage version (5.0.0)

• My installed MongoDb version is (4.4.2)

• I have Multer (1.4.2) installed to upload files

• The Node version used to run the code is (1.4.2)

• I'm not using Mongoose connection objects to create storage instances.

To Reproduce

1. upload folder with formdata. (I can confirm that each file contains webkitRelativePath)
2. on nodejs file hanlder, only see the filename instead of the relative path

Expected behavior

Describe the bug
Version 5.0.2 overwrites mongoDB/mongoose types that break other mongoDB/mongoose-dependent packages, like connect-mongo. Not sure which set of types are actually doing the harm.

Environment

• I'm using multer-gridfs-storage version 5.0.2.
• My installed MongoDb version is 4.81. <-- I do not need this in this package, and simply installed it to see if it would address the issue. It did not.
• I have Multer 1.4.4-lts.1 installed to upload files. <-- also tried 1.4.5-lts.1, no luck.
• The Node version used to run the code is 16.14.0.

To Reproduce
npm i connect-mongo -> package.json shows "connect-mongo": "^4.6.0";
npm i [email protected] -> package.json shows "multer": "^1.4.4-lts.1";
npm i multer-gridfs-storage -> npm shows warnings/errors addressed in #490 , package.json shows "multer-gridfs-storage": "^5.0.2".

Upon build, the following errors are generated:

tsc

node_modules/connect-mongo/build/main/lib/MongoStore.d.ts:2:55 - error TS2305: Module '"mongodb"' has no exported member 'WriteConcernSettings'.

2 import { Collection, MongoClient, MongoClientOptions, WriteConcernSettings } from 'mongodb';
~~~~~~~~~~~~~~~~~~~~

node_modules/mongoose/types/aggregate.d.ts:22:25 - error TS2694: Namespace '"C:/Users/servi/Desktop/d-drive/cctx/oba-express/node_modules/@types/mongodb/index"' has no exported member 'CollationOptions'.

22 collation?: mongodb.CollationOptions;
~~~~~~~~~~~~~~~~

node_modules/mongoose/types/aggregate.d.ts:36:23 - error TS2694: Namespace '"C:/Users/servi/Desktop/d-drive/cctx/oba-express/node_modules/@types/mongodb/index"' has no exported member 'ExplainVerbosityLike'.

36 explain?: mongodb.ExplainVerbosityLike;

And 60 more...The build command output ends with this summary:

Found 63 errors in 11 files.

Errors Files
1 node_modules/connect-mongo/build/main/lib/MongoStore.d.ts:2
9 node_modules/mongoose/types/aggregate.d.ts:22
3 node_modules/mongoose/types/collection.d.ts:8
4 node_modules/mongoose/types/connection.d.ts:75
1 node_modules/mongoose/types/error.d.ts:37
1 node_modules/mongoose/types/index.d.ts:467
7 node_modules/mongoose/types/indexes.d.ts:18
26 node_modules/mongoose/types/models.d.ts:23
8 node_modules/mongoose/types/query.d.ts:103
2 node_modules/mongoose/types/schemaoptions.d.ts:43
1 node_modules/mongoose/types/session.d.ts:4

Expected behavior
Upon build, no errors generated. When this package is removed only, build runs perfectly as expected.

Hello, I have tried to use multer-gridfs-storage and got the warning mentioned in the title. Could someone help?

upload.js

const multer = require('multer');
const { GridFsStorage } = require('multer-gridfs-storage');

const MONGO_URL = process.env.MONGO_URL;
const DATABASE = process.env.DATANASE;
const IMAGE_BUCKET = process.env.IMAGE_BUCKET;

const storage = new GridFsStorage({
  url: MONGO_URL + DATABASE,
  options: { useNewUrlParser: true, useUnifiedTopology: true },
  // file is the function to control the file storage in the database
  file: (req, file) => {
    const match = ['image/png', 'image/jpeg'];

    if (match.indexOf(file.mimetype) === -1) {
      const filename = `${Date.now()}-J4I-${file.originalname}`;
      return filename;
    }

    return {
      bucketName: IMAGE_BUCKET,
      filename: `${Date.now()}-J4I-${file.originalname}`,
    };
  },
});

// Multer module initializes middleware
const uploadFiles = multer({ storage: storage }).single('file');

// util.promisify() makes the exported middleware object can be used with async/await
const uploadFilesMiddleware = util.promisify(uploadFiles);
module.exports = uploadFilesMiddleware;

.env

## MongoDB
MONGO_URL='mongodb://localhost:27017/'
DATABASE='J4I_files_db'
IMAGE_BUCKET='photos'

Hi,

I'm trying to control beforehand if a file already exists in the GridFS according to its metadata and if it exists, update de old one. But as all the uploading work is done in the storage definition I'm not sure about how to proceed.

What I'm doing as a workarond is to upload the file and then remove the rest of files with the same metadata, but it's too much tricky and I think that it must be possible to control if it exists before. The question is when/where?

Here is my code:

server.js

const storage = new GridFsStorage({
    url: db.url,
    file: (req, file) => {
        return new Promise((resolve, reject) => {
            crypto.randomBytes(16, (err, buf) => {
                if (err) {
                    return reject(err);
                }
                const body = JSON.parse(JSON.stringify(req.body));
                const str_meta = JSON.parse(JSON.stringify(body.metadata));
                const meta = JSON.parse(str_meta);
                const filename = buf.toString("hex") + path.extname(file.originalname);
                const fileInfo = {
                    filename: filename,
                    metadata: meta,
                    bucketName: "uploads"
                };

                resolve(fileInfo);
            });
        });
    }
});

const upload = multer({
    storage
});

router.js

module.exports = (app, upload, gfs, mongoose) => {

    const files = require("../controllers/files.controller.js");

    var router = require("express").Router();

    // Upload file
    router.post("/", upload.single("file"), files.upload(gfs, mongoose));

};

controller.js

// Upload new file to GridFS
exports.upload = (gfs, mongoose) => {
    return function (req, res, next) {

        const body = JSON.parse(JSON.stringify(req.body));
        const str_meta = JSON.parse(JSON.stringify(body.metadata));
        const meta = JSON.parse(str_meta);

        // once document uploaded, check if existed before and remove old one(s)
        gfs
            .find({
                'metadata.id': meta.id, 'metadata.type': meta.type
            })
            .toArray((err, files) => {

                if (!files || files.length === 0) {
                    return res.status(404).json({
                        err: "No file(s) found"
                    });
                }

                if (files.length === 1) {
                    return res.send({
                        message: "Document successfully uploaded!",
                        file: req.file
                    });
                }

                const sortedFiles = files.sort((a, b) => b.uploadDate - a.uploadDate)
                sortedFiles.shift()

                const idsArray = sortedFiles.map(a => a._id);
                // it seems that gfs hasn't deleteMany method, so remove files one by one
                idsArray.forEach(item => {
                    gfs.delete(new mongoose.Types.ObjectId(item), (err, data) => {
                        if (err) return res.status(404).json({ err: err.message });
                    })
                });

                res.send({
                    message: "Document successfully updated!",
                    file: req.file
                });

            })
    }

};

Thanks!

Describe the bug
When used with MongoDB v5, the following deprecation warning is displayed due to the usage of isConnected() within multer-gridfs-storage:

(node:19016) DeprecationWarning: isConnected is deprecated and will be removed in the next major version

Environment

• I'm using multer-gridfs-storage version 5.0.2

• My installed MongoDb version is 5.0.8

• I have Multer 1.4.3 installed to upload files

• The Node version used to run the code is 17.4.0

To Reproduce
Simply create a new GridFsStorage in node.
I traced the depreciation from the new GridFsStorage() call (not including parameters because they're irrelevant to the deprecation):

    at GridFsStorage._updateConnectionStatus (myapp/node_modules/multer-gridfs-storage/lib/gridfs.js:407:33)
    at GridFsStorage._setDb (myapp/node_modules/multer-gridfs-storage/lib/gridfs.js:446:14)
    at GridFsStorage._connect (myapp/node_modules/multer-gridfs-storage/lib/gridfs.js:335:18)
    at new GridFsStorage (myapp/node_modules/multer-gridfs-storage/lib/gridfs.js:91:14)
    at singleFileUpload (myapp/api/src/routes/file.js:46:21)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)

The warning occurs because of the isConnected() call in the following screenshot:

The warning is thrown within myapp/node_modules/mongodb/lib/mongo_client.js:

Expected behavior
Not showing the warning/the package no longer being deprecated

I have been making use of the metadata array in order to store user id's so that when I retrieve the files, I search by user id.

This works a treat for uploading jpeg and jpg, but in testing PNG, my metadata.user_id is set to null, each and every time.

Is there a known issue with PNGs? The file is uploaded to the mongodb perfectly fine and I can see the document there, but the metadata.user_id is set to null. I've done various troubleshooting and I am definitely sending and receiving the user_id when making the request, just the metadata is ignoring it.

Hi, req.body is undefined inside the metadata function.

metadata: function(req, file, cb) {
        let parsedMime = contentType.parse(file.mimetype);
        let meta = {};
        console.log(req.body); //undefined
}

But I can access it inside the upload function.

router.post('/upload', function(req, res){
    let mongoId = new mongoose.Types.ObjectId();
    req.mongoId = mongoId;
    uploadsingle(req, res, (err)=>{
      console.log(req.body);  //defined
      if(err){
        return utils.handleErrResp(req, res, err);
      }
      return res.json({
        responseCode: messages.defaultSuccessObject.responseCode,
        responseMessage: messages.defaultSuccessObject.responseMessage
      });
    });
});

Issue checklist

I'm submitting a...

• Bug report
• Feature request
• Documentation issue or request
• Support request
• Regression

Environment

• I'm using multer-gridfs-storage version (major.minor.patch)

• My installed MongoDb version is (major.minor.patch)

• I have Multer (major.minor.patch) installed to upload files

• I'm (using/not using) Mongoose connection objects to create storage instances. The Mongoose version installed is (major.minor.patch)

• The Node version used to run the code is (major.minor.patch)

Current behavior

Expected behavior

Am using array method to upload multiple file one being the video while the other being the thumbnail of the video. How can i store the thumbnail to that particular file?. I was thinking of putting the thumbnail in meta data is it possible ? or is there any way to upload the file and the thumbnail as single array ?

Describe the bug

When I tried to install multer-gridfs-storage with the latest lts version of multer, it throws an error

npm ERR! code ERESOLVE
npm ERR! ERESOLVE unable to resolve dependency tree
npm ERR! 
npm ERR! While resolving: [email protected]
npm ERR! Found: [email protected]
npm ERR! node_modules/multer
npm ERR!   multer@"^1.4.5-lts.1" from the root project
npm ERR! 
npm ERR! Could not resolve dependency:
npm ERR! peer multer@"^1.4.2" from [email protected]
npm ERR! node_modules/multer-gridfs-storage
npm ERR!   multer-gridfs-storage@"*" from the root project
npm ERR! 
npm ERR! Fix the upstream dependency conflict, or retry
npm ERR! this command with --force, or --legacy-peer-deps
npm ERR! to accept an incorrect (and potentially broken) dependency resolution.
npm ERR! 
npm ERR! See /home/ubuntu/.npm/eresolve-report.txt for a full report.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/ubuntu/.npm/_logs/2022-10-05T05_29_53_065Z-debug-0.log

Environment

• I'm using multer-gridfs-storage version ^5.0.2

• My installed MongoDb version is ^4.9.1

• I have Multer 1.4.5-lts.1 installed to upload files

• The Node version used to run the code is 16.17.1

• I'm using Mongoose connection objects to create storage instances. The Mongoose version installed is ^6.6.3

To Reproduce

• update multer dependency to 1.4.5-lts.1

• Bump multer dependency to 1.4.5-lts.1 #492

Expected behavior

• installing multer-gridfs-storage should not throw any error

Hi
I'm planning to use this library (http://sharp.dimens.io/en/stable/) to provide file resize before saving into GridFS. It's possible with your library? I mean something like a transform function.
Thanks

Describe your problem

README.md states: Compatibility with MongoDb versions 2 and 3.

Please clarify - is it compatible with MongoDb v4+ ?
Is there a better way to do things with MongoDb v4+?

Environment

Current version

Additional context

Issue checklist

I'm creating a video on demand Api I want to add Movie Title, Description and category property to my file property

I'm submitting a...

• Bug report
• Feature request
• Documentation issue or request
• Support request
• Regression

Environment

• I'm using multer-gridfs-storage version (major.minor.patch)

• My installed MongoDb version is (major.minor.patch)

• I have Multer (major.minor.patch) installed to upload files

• I'm (using/not using) Mongoose connection objects to create storage instances. The Mongoose version installed is (major.minor.patch)

• The Node version used to run the code is (major.minor.patch)

Current behavior

Expected behavior

do you use file system underneath for stream files?
in stackoverflow everyone use file system for create stream file like:
fs.createReadStream(uploadFilePath)
do you use this too?

Hello!

I get following error if I try to use the url option with a environment variable. (working with dotenv module)

Error creating storage engine. At least one of url or db option must be provided

The variable itself is correct and works when I try to connect with mongoose.

const storage = new GridFsStorage({
  url: process.env.MONGO_URI,
  options: {
    useNewUrlParser: true
  },
  file: (req, file) => {
    return new Promise((resolve, reject) => {
      crypto.randomBytes(16, (err, buf) => {
        if (err) {
          return reject(err)
        }
        const filename = buf.toString('hex') + path.extname(file.originalname)
        const fileInfo = {
          filename: filename,
          bucketName: 'uploads'
        }
        resolve(fileInfo)
      })
    })
  }
})

Isn't it possible to do so or am I doing something wrong here?

Describe the bug
multer-gridfs-storage uses the deprecated @types/mongodb package as a dependency. There are conflicts with the mongodb package in version >= 5:

node_modules/multer-gridfs-storage/node_modules/@types/mongodb/index.d.ts:58:74 - error TS2724: '"bson"' has no exported member named 'ObjectID'. Did you mean 'ObjectId'?

58 export { Binary, DBRef, Decimal128, Double, Int32, Long, MaxKey, MinKey, ObjectID, ObjectId, Timestamp } from "bson";
                                                                            ~~~~~~~~

  node_modules/bson/bson.d.ts:918:22
    918 export declare class ObjectId extends BSONValue {
                             ~~~~~~~~
    'ObjectId' is declared here.


Found 1 error in node_modules/multer-gridfs-storage/node_modules/@types/mongodb/index.d.ts:58

Environment

• I'm using multer-gridfs-storage package version 5.0.2 with mongodb package version 5.0.1

• My installed MongoDb version is 6.0.1

• I have Multer 1.4.5-lts.1 installed to upload files

• The Node version used to run the code is 16.17.0

• I'm using Mongoose connection objects to create storage instances. The Mongoose version installed is 6.9.1

To Reproduce
Integrate the above packages into any project and try to build the project.

Expected behavior
Since mongodb now comes with its own types, the deprecated @types/mongodb package can be dispensed with altogether.

While 3.1 works perfectly, 3.2.x for me gets:

TypeError: this.db.on is not a function
    at GridFSStorage._setDb (/Users/malix/Temp/nz/node_modules/multer-gridfs-storage/lib/gridfs.js:253:6)
    at MongoClient.connect (/Users/malix/Temp/nz/node_modules/multer-gridfs-storage/lib/gridfs.js:209:12)
    at connectCallback (/Users/malix/Temp/nz/node_modules/mongodb/lib/mongo_client.js:515:5)
    at /Users/malix/Temp/nz/node_modules/mongodb/lib/mongo_client.js:449:13
    at _combinedTickCallback (internal/process/next_tick.js:131:7)
    at process._tickCallback (internal/process/next_tick.js:180:9)

connectiing with:

import * as GridFsStorage from 'multer-gridfs-storage';
const storage = new GridFsStorage({
    url: config.secrets.uris,
    //...
});```

!?

database connection file

const mongoose = require('mongoose');
const multer = require('multer')
const {GridFsStorage} = require('multer-gridfs-storage')
const Grid = require('gridfs-stream');
const path = require('path');
const dotenv = require('dotenv');
const crypto = require('crypto')
dotenv.config( { path : 'config.env'} )
let storage;
exports.connectDB = async ()=>{
    try{
        mongoose.connect(process.env.MONGO_URL,{
            useNewUrlParser:true,
            useUnifiedTopology:true,

        }).then(
          (promise)=>{
            console.log(`Mongodb connected:${promise.connection.host}`)
            // console.log(`MongoDb connected:${promise.connection.db}`)
            const gfs = Grid(promise.connection.db, mongoose.mongo);
            gfs.collection('uploads');
            
            storage = new GridFsStorage({
              url:process.env.MONGO_URL,
              db:promise,
              file: (req, file) => {
                return new Promise((resolve, reject) => {
                  crypto.randomBytes(16, (err, buf) => {
                    if (err) {
                      return reject(err);
                    }
                    const filename = buf.toString('hex') + path.extname(file.originalname);
                    const fileInfo = {
                      filename: filename,
                      bucketName: 'uploads'
                    };
                    resolve(fileInfo);
                  });
                });
              }
            });
            console.log(storage)
          },
          (err)=>{
            console.log(err)
          }
        )
    }catch(err){
        console.log(err);
        process.exit(1);
    }
} 
exports.upload = multer({storage})

post request with upload as middleware

route.post('/example',database.upload.single('file'),async (req,res)=>{
    res.json({file:req.file})
})

when I do a post request with postman there's no error but file doesn't upload on MongoDB neither the upload collection is being created.

and output of console.log(storage)

GridFsStorage {
  _events: [Object: null prototype] {},
  _eventsCount: 0,
  _maxListeners: 0,
  db: Db {
    s: {
      client: [MongoClient],
      options: [Object],
      logger: [Logger],
      readPreference: [ReadPreference],
      bsonOptions: [Object],
      pkFactory: [Object],
      readConcern: undefined,
      writeConcern: [WriteConcern],
      namespace: [MongoDBNamespace]
    }
  },
  client: null,
  connected: true,
  connecting: false,
  caching: false,
  error: null,
  configuration: {
    url: 'my_mongodb_url(removed original one)',
    db: <ref *1> Mongoose {
      connections: [Array],
      models: [Object],
      events: [EventEmitter],
      options: [Object],
      _pluralize: [Function: pluralize],
      Schema: [Function],
      model: [Function (anonymous)],
      plugins: [Array],
      default: [Circular *1],
      mongoose: [Circular *1]
    },
    file: [Function: file]
  },
  _file: [Function: file],
  _options: undefined,
  [Symbol(kCapture)]: false
}

Issue checklist

I'm submitting a...

• Bug report
• Feature request
• [x ] Documentation issue or request
• [ x] Support request
• Regression

Environment

• I'm using multer-gridfs-storage version (major.minor.patch)

• My installed MongoDb version is (major.minor.patch)

• I have Multer (major.minor.patch) installed to upload files

• I'm (using/not using) Mongoose connection objects to create storage instances. The Mongoose version installed is (major.minor.patch)

• The Node version used to run the code is (major.minor.patch)
  v8.11.3

Current behavior

I'm using this library to backend for my video on demand Api (for handling file upload). File upload is successful, the problem i'm facing is i want upload to two files as a single Array one being the Video file and the other the thumbnail or Poster of the movie, i tried using upload.array but is not uploading as single array that contains the two file. The code for files upload

How the are File in the Db

Expected behavior

Upload video file with its poster or thumbnail as single array like

[{Movie}, {Poster/Thumbnail here}]

Good job devs, but it would be greater if somebody could add @types/multer-gridfs-storage to work with Typescript.

When run GridFsStorage, it's return the "TypeError: mongodb_1.ObjectID is not a constructor
at Function. (C:\dev\api-cariocamix\node_modules\multer-gridfs-storage\src\gridfs.ts:146:18)
at Generator.next ()
at fulfilled (C:\dev\api-cariocamix\node_modules\multer-gridfs-storage\lib\gridfs.js:5:58)
at processTicksAndRejections (node:internal/process/task_queues:96:5) {
storageErrors: []
}".

Bug is in: ./lib/gridfs.js, line 125

Thanks for great job!

Best regards,
Pedro Brennon
[email protected]

Describe the bug

Environment

• I'm using multer-gridfs-storage version (major.minor.patch)

• My installed MongoDb version is (major.minor.patch)

• I have Multer (major.minor.patch) installed to upload files

• The Node version used to run the code is (major.minor.patch)

• I'm (using/not using) Mongoose connection objects to create storage instances. The Mongoose version installed is (major.minor.patch)

To Reproduce

Expected behavior

Describe the bug
After updating mongodb and mongoose packages, when I try to create new GridFsStorageInstance, this error appears:
Type 'Db' is not assignable to type 'DbTypes | Promise<DbTypes>'.

The full class is:

import { Injectable } from '@nestjs/common';
import { InjectConnection } from '@nestjs/mongoose';
import {
  MulterModuleOptions,
  MulterOptionsFactory,
} from '@nestjs/platform-express';
import { Connection } from 'mongoose';
import { GridFsStorage } from 'multer-gridfs-storage/lib/gridfs';

@Injectable()
export class GridFsMulterConfigService implements MulterOptionsFactory {
  private gridFsStorage: GridFsStorage;

  public constructor(@InjectConnection() private connection: Connection) {
    this.gridFsStorage = new GridFsStorage({
      db: this.connection.db,         // Type 'Db' is not assignable to type 'DbTypes | Promise<DbTypes>
      file: async (_, file) => {
        return new Promise(async (resolve) => {
          const filename = file.originalname.trim();
          const fileInfo = {
            filename,
          };

          resolve(fileInfo);
        });
      },
    });
  }

  public createMulterOptions(): MulterModuleOptions {
    return {
      storage: this.gridFsStorage,
    };
  }
}

Environment

• I'm using multer-gridfs-storage version 5.0.2

• My installed MongoDb version is 4.12.0

• I have Multer 1.4.5-lts.1 installed to upload files

• The Node version used to run the code is 16.16.0

• I'm (using/not using) Mongoose connection objects to create storage instances. The Mongoose version installed is (major.minor.patch)

I'm submitting a...

• Bug report
• Feature request
• Documentation issue or request
• Support request
• Regression

Environment

• I'm using multer-gridfs-storage version (^3.2.2)

• My installed MongoDB version is (v3.4.6)

• I have Multer (^1.3.1) installed to upload files

• The Node version used to run the code is (v10.1.0)

Current behavior

req.body is empty in file(req, file) callback, when the multipart request contains texts fields after the file filed, but when the field comes before the file, it gets passed to the req.body

Expected behavior

req.body should contain text fields, it does not matter whether the file comes before or after fields

Please note, this may be a very dumb issue request. If you have seen a solution just link it below.

Unable to prevent certain types of files from being saved in the DB.

I am trying to only save images under the size of 5MB
I am working on building an express API using NodeJS. I want to make sure that the client side of this app only submits photo's to the backend.
But the issue I am facing is that file configuration function always saves a file to the database even if you pass an undefined or null object.

As you can see below. I was hoping to only save images but no matter what the input it always saves the file.

const storage = new GridFsStorage({
    url: MONGO_URI,
    options: {
        useNewUrlParser: true,
        useUnifiedTopology: true
    },
    file: (req, file) => {
        if (file.contentType.includes('image/')) {
            return file;
        }
    }
});

const upload = multer({ storage });

I am not sure if this is the right approach for saving files using multer-gridfs-storage but this is the only one I could find.

Environment
I am using a Windows machine with WSL installed.
My Node Version -> v12.18.1
Package version -> "multer-gridfs-storage": "^4.2.0"

I am using mongoose for connecting and my guess that Im not using the library correctly because of that , maybe you could assist with my issue.

my connection looks like this:

mongoose.connect('mongodb://foo123.mlab.com:12345/database', {
    auth: {
      user: 'username',
      password: 'password'
    }
  })
  .then(() => console.log('connection successful'))
  .catch((err) => console.error(err));

and my multer-gridfs-storage function looks like that:

const storage = require('multer-gridfs-storage')({
  url: 'mongodb://foo123.mlab.com:12345/database'
});


const upload = multer({storage: storage})

router.post('/', upload.single('carImage') ,(req, res) => {
    console.log('trying to upload image')
    console.log(req.file);    
    let todayArr = today.slice(0, 4);

})

Where am I wrong with my approach?

How I can read file from storage?

First I want to thank you for this wonderful package.

Do you know when this project will be updated to use Mongoose v6 and MongoDB v4? I see that PRs for these issues are already open:
#375
#376

I have a project that relies on both multer-gridfs-storage and mongoose, and I can't update to mongoose v6 until this package is updated due to type conflicts.

Describe your problem

Environment

Additional context

Is your feature request related to a problem? Please describe.

Although already discussed and closed in #5 I believe it might be worth another thought: eg if you had the requirement to encrypt data, but do not have the enterprise version, it would be quite fancy to do that with a transformstream, eg as described here: http://codewinds.com/blog/2013-08-20-nodejs-transform-streams.html

Describe the solution you'd like

in fromMulterStream enhance this line pump([readStream, writeStream]) to:

let streamArray = [readStream]
if (Array.isArray(streamOptions.streamTransformers) && streamOptions.streamTransformers?.length) {
  streamArray = streamArray.concat(streamOptions.streamTransformers)
}
streamArray = streamArray.concat([writeStream])
pump(streamArray)

Describe alternatives you've considered

Alternative would be as described in #5 to re-read the data transform, it and write it again and then delete the old data.

Additional context

If you think that's worth thinking of I could provide a PR.

Hi there,

I just ran into this warning after updating to v.3.1.0.

(node:1954) DeprecationWarning: GridStore is deprecated, and will be removed in a future version. Please use GridFSBucket instead

Describe the bug
When using multer version ^1.4.5-lts.1 as well as the latest version of this plugin we get the following error

npm WARN   multer@"^1.4.5-lts.1" from the root project
npm WARN 
npm WARN Could not resolve dependency:
npm WARN peer multer@"^1.4.2" from [email protected]
npm WARN node_modules/multer-gridfs-storage
npm WARN   multer-gridfs-storage@"^5.0.2" from the root project
npm WARN 
npm WARN Conflicting peer dependency: [email protected]
npm WARN node_modules/multer
npm WARN   peer multer@"^1.4.2" from [email protected]
npm WARN   node_modules/multer-gridfs-storage
npm WARN     multer-gridfs-storage@"^5.0.2" from the root project

Environment
"multer": "^1.4.5-lts.1",
"multer-gridfs-storage": "^5.0.2",

• I'm using multer-gridfs-storage version "multer": "^1.4.5-lts.1",

• My installed MongoDb version is "mongodb": "^4.5.0",

• I have Multer ^1.4.5-lts.1 installed to upload files

• The Node version used to run the code is v17.9.0

• I'm using Mongoose connection objects to create storage instances. The Mongoose version installed is ^6.3.5

To Reproduce
Create a project with this as your package.json

{
  "name": "projectname",
  "version": "0.1.0",
  "private": true,
  "scripts": {
    "dev": "next dev",
    "build": "next build",
    "start": "next start",
    "lint": "next lint"
  },
  "dependencies": {
    "@next-auth/mongodb-adapter": "^1.0.3",
    "@turf/circle": "^6.5.0",
    "@turf/turf": "^6.5.0",
    "axios": "^0.27.2",
    "bootstrap": "^5.1.3",
    "bootstrap-daterangepicker": "^3.1.0",
    "crypto-js": "^4.1.1",
    "d3-request": "^1.0.6",
    "date-fns": "^2.28.0",
    "formidable": "^2.0.1",
    "gridfs-stream": "^1.1.1",
    "jquery": "^3.6.0",
    "mapbox-gl": "^2.8.2",
    "micro": "^9.3.4",
    "moment": "^2.29.3",
    "mongodb": "^4.5.0",
    "mongoose": "^6.3.5",
    "multer": "^1.4.5-lts.1",
    "multer-gridfs-storage": "^5.0.2",
    "multiparty": "^4.2.3",
    "next": "12.1.5",
    "next-auth": "^4.3.1",
    "next-connect": "^0.12.2",
    "react": "18.0.0",
    "react-bootstrap": "^2.3.1",
    "react-bootstrap-daterangepicker": "^7.0.0",
    "react-dom": "18.0.0",
    "react-map-gl": "^7.0.12",
    "recharts": "^2.1.10",
    "uuid": "^8.3.2"
  },
  "devDependencies": {
    "eslint": "8.13.0",
    "eslint-config-next": "12.1.5"
  }
}

Expected behavior
no error or warnings.

Issue most likely can be resolved simply by recompiling the TypeScript.

Issue is located in line 125 of gridfs.js

if (!hasId) {
                previous.id = new mongodb_1.ObjectId(); // should be ObjectID instead
}

Describe the bug

MongoDB NodeJS driver 4.0 breaks this library.

I don't think this is a major issue, just needs an update.

Environment

• I'm using multer-gridfs-storage version 5.0.0

• My installed MongoDb version is 4.4.6 (Docker)

• I have Multer 1.4.2 installed to upload files

• The Node version used to run the code is 12.21.0

To Reproduce

Literally try do anything

Expected behavior

Upload succeeds

Hello,

I am using multer-gridfs-storage v4.03. It works fine in my AWS Cloud9 dev environment however I am getting an issue requiring the package in my production environment on an AWS lightsail environment using NGINX reverse proxy and pm2.

The error I am getting is in the gridfs.js file

        async    _resolveConnection() {
1|API    |       ^^^^^^^^^^^^^^^^^^
1|API    | SyntaxError: Unexpected identifier
1|API    |     at createScript (vm.js:56:10)
1|API    |     at Object.runInThisContext (vm.js:97:10)
1|API    |     at Module._compile (module.js:549:28)
1|API    |     at Object.Module._extensions..js (module.js:586:10)
1|API    |     at Module.load (module.js:494:32)
1|API    |     at tryModuleLoad (module.js:453:12)
1|API    |     at Function.Module._load (module.js:445:3)
1|API    |     at Function._load (/usr/lib/node_modules/pm2/node_modules/@pm2/io/build/mai
n/metrics/httpMetrics.js:172:43)

Environment

-I'm using multer-gridfs-storage version 4.03"

• I am connecting to a MongoDB Atlas connection version 4.2.6

• I have Multer 1.4.2 installed to upload files

• The Node version used to run the code is 14

• I'm using Mongoose connection objects to create storage instances. The Mongoose version installed is 5.7.7

Any help would be appreciated. Thanks

Issue checklist

I'm submitting a...

• Bug report
• Feature request
• Documentation issue or request
• Support request
• Regression

Environment

• I'm using multer-gridfs-storage version (major.minor.patch)

• My installed MongoDb version is (major.minor.patch)

• I have Multer (major.minor.patch) installed to upload files

• I'm (using/not using) Mongoose connection objects to create storage instances. The Mongoose version installed is (major.minor.patch)

• The Node version used to run the code is (major.minor.patch)
  v8.11.3

Current behavior

Am uploading file to the my mongodb using multergridfs storage i created a movieSchema with fileID property which i reference with id of the file i'm uploading, the fileID is reference is successfully with file id the problem is when try populate method am not getting file property.

Expected behavior

Populate the Movie schema with file properties

Movie schema
`const mongoose = require('mongoose');
const Schema = mongoose.Schema;

const MovieSchema = new Schema({
description: String,
category: String,
token: {
type: String,
unique: true,
expires: '30s'
},
fileID: [{
type: mongoose.Schema.Types.ObjectId,
ref: 'fs.files'
}],
posterID: {
type: mongoose.Schema.Types.ObjectId,
ref: 'fs.files'
}
});

const Movie = mongoose.model('Movies', MovieSchema);

module.exports = Movie;`

file upload
`router.post('/', upload.array('file', 'file'), (req, res) => {
console.log(req.files)
const movie = new Movie({
description: req.body.Description,
category: req.body.Category,
token: token,
fileID: [{
type: req.files[0].id,
ref: 'fs.files'
}],
posterID: [{
type: req.files[1].id,
ref: 'fs.files'
}]
});
console.log(movie)
movie.save(function(err) {
if (err) {
console.log(err);
return;
}

    res.json({
        "success": "true"
    });
});

});

router.get('/movies', (req, res) => {
Movie
.find({
fileID: '5c0ac602105b096fa1688577'
})
.populate('fileID', 'fieldname').then(files => {
res.send(files)
})
.catch(err => {
res.send(err)
})
})

module.exports = router;`