vsivsi / gridfs-locking-stream Goto Github PK

Hi, first of all, thanks for this great library. I'm currently trying to get it working in a restify server, piping the GridFS stream directly to the response.

However, I get the exception "Error: Lock.releaseLock Lock document not found in collection" after the data was successfully written to the response and read by the client. I tracked it down to two lock.releaseLock calls at index.js#L176 and index.js#L180. The stream emits first end and then close right after, leading to two releaseLock calls. The first releaseLock call successfully updates the locks collection while the second fails with the mentioned exception because it can't find the lock with read_locks > 0 anymore.

The close event only happens when specifying a endPos in the range of bytes to read in the createReadStream options. When reading the whole file, only the end event is emitted, only one releaseLock call is made and it works fine. I'm thinking the gridfs-stream library emits these events differently depending on whether an endPos is given.

I can work around the issue for now by removing the close listener from the ReadStream, thus avoiding the second releaseLock call. But I'm not sure if that can cause other issues so I'd appreciate if you look into this issue.

I have been using "gridfs-stream" for quite some time now and have just started experiencing "MongoError: no chunks found for file, possibly corrupt" issues which I am very sure is due to concurrent sessions.

when using "gridfs-locking-stream", the collection I have been using is "uploads". When I try to pass this with the "options" it just doesn't find anything.

I have tried including a specific _id that I do get back when using "gridfs-stream" but it still doesn't locate the document.

This is being used together with mongoose, express and multer. Any help would be appreciated.

an example:

`const express = require('express');
const router = express.Router();
const upload = require('../config/upload');
const UploadCollection = require('../models/uploadModel');
const User = require('../models/user');
const mongoose = require('mongoose');
// const Grid = require('gridfs-stream');
const Grid = require('gridfs-locking-stream');
const config = require('../config/database');
const sharp = require('sharp');
const passport = require('passport');

// Create mongo Connection
Grid.mongo = mongoose.mongo;
const conn = mongoose.createConnection(config.database);

// Init gfs
let gfs;

conn.once('open', () => {
// init Stream
gfs = Grid(conn.db, mongoose.mongo);
// gfs.collection('uploads');
});

// @route Get /
// @desc Loads form
router.get('/', (req, res) => {
const options = {
_id: '5b5d8eae569a08653a710e3d',
root: 'uploads',
};
gfs.files.find(options).toArray((err, files) => {
// check if files exist
if (err) {
console.log(err);
return res.status(404).json({ success: false, msg: err });
}
if (!files || files.length === 0) {
// return res.status(404).json({ success: false, msg: 'No files exist' });
return res.json({ success: false, msg: 'No files were found' });
} else {
files.map(file => {
if (file.contentType === 'image/jpeg' || file.contentType === 'image/jpg' || file.contentType === 'image/png') {
file.isImage = true;
} else {
file.isImage = false;
}
});
return res.json({ success: true, files: files });
}
});
});`

Hi,

This is more of a call for help than an issue, as i've been seeing this error for the past 5 days and my headache level reached critical today :)

I don't know if this is an issue with the module or with my code, but i've checked everything and Google doesn't seem to be of any help at all in this case.

This used to work before updating to v1.x. No changes were made to my code.

From what i see in the module, files_id should never be null if i specify _id in the options object, which i do, but it crashes when creating a new LockCollection, at index.js:53.

My code is as follows:

var gfs = new Grid(mongoose.connection.db, mongoose.mongo, collection),
        options = {
            _id: new mongoose.Types.ObjectId(),
            mode: 'w',
            filename: attachmentDetails.filename,
            content_type: attachmentDetails.mimeType,
            root: 'attachments',
            metadata: {}
        },
        result = {
            sequence_id: options._id,
            file_name: options.filename,
            mime_type: options.content_type,
            size: buffer.length,
            metadata: {}
        };

// Meta processing stuffs

gfs.createWriteStream(options, function (err, writeStream) {
        if (err) {
            logHelper.error('db', new Error(err), options);

            return callback(err);
        }

        if (writeStream) {
            var dataStream = streamifier.createReadStream(buffer);

            dataStream.pipe(writeStream);

            return callback(null, result);
        } else {
            return callback({
                code: 500,
                message: 'A write lock could not be obtained'
            })
        }
    });

Do you see anything wrong with it?

Thanks in advance for your help and thanks for a great module.

Current package.json:

{
  "dependencies": {
    "gridfs-stream": ">= 0.5.3",
    "gridfs-locks": ">= 1.2.2"
  }
}

is invalid because current gridfs-locks-stream is probably not compatible with breaking changes that will occure in gridfs-locks 2.x, and gridfs-stream 1.x or 2.x...

It should probably be something like that, using "semver compatible" constraints:

{
  "dependencies": {
    "gridfs-stream": "^0.5.3",
    "gridfs-locks": "^1.2.2"
  }
}

Please check the link. I discussed the question there. I think gridfs-locking-stream is not compatible with the new version of mongoose.
https://stackoverflow.com/questions/49325749/mongoose-and-gridfs-locking-stream-interaction