threez / file-queue Goto Github PK
View Code? Open in Web Editor NEWFile system based queue for node.js
File system based queue for node.js
I had to modify the length function in order to get it to return correct values the way I am using it.
fs.stat ==> ulink was always reporting 2.
// Finds the length of the queue (list of files in new)
Maildir.prototype.length = function(callback) {
fs.readdir(this.dirPaths[NEW]+'/', function(err, files) {
if (err) { callback(err); }
else { callback(null, files.length); }
});
};
It seems that they are not created automatically, nevertheless the program doesn't crash
Hi,
I implemented a simple example using the file-queue. I don't understand how I should process the queue. When I run "pop" it will just take one message from the queue and processes it.
So I understand that I need to take care of the queue itself, meaning that I need to check the length constantly and run pop by my own logic. It's not like pop is called whenever there is a new file in the queue, right?
Thanks for clarifying.
I'm trying to understand why/how this is "lock free" - what's the exact mechanism that makes it "lock free"?
thanks!
Hello,
how do you think is it possible to optimize code:
Line 90 in 6f2bfa6
Maildir.prototype.listNew = function(callback) {
this.fs.readdir(this.dirPaths[NEW], callback);
};
the point is for each pop
operation it reads all content of the directory.
pop()->readdir()
pop()->readdir()
pop()->readdir()
In the sequence code
Line 76 in 6f2bfa6
tryPop
you use only 1 item from the array.
If the queue has size 1^e6 messages it .....
How about to optimize this place?
For example
memoize
listNew()
until it has elements?Hello,
Problematic code is:
Line 58 in 6f2bfa6
that.watcher = that.fs.watch(that.dirPaths[NEW], {}, function(err, messages) {
that.emit('new', [messages]);
});
This code doesn't work for NFS3.
http://docs.libuv.org/en/v1.x/fs_event.html
UV_FS_EVENT_WATCH_ENTRY = 1,
/*
* By default uv_fs_event will try to use a kernel interface such as inotify
* or kqueue to detect events. This may not work on remote file systems such
* as NFS mounts. This flag makes fs_event fall back to calling stat() on a
* regular interval.
* This flag is currently not implemented yet on any backend.
*/
Please, add polling: say for each 30-60 seconds fs.readdir -> if files are present emit 'new' event.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.