apache / nano Goto Github PK

Ability to pipe a document or attachment to a stream

Have you seen the library for CouchDB transactions I released yesterday, Txn?

The idea is, instead of having a document and trying to store it, you have an id and you first fetch, then see what it looks like, make some changes, and finally store again. (With automatic retry features.) However, the goal is that your code looks roughly the same.

Do you think it would make sense for Nano to expose this functionality? If so, what do you think the API would look like?

Support bulk load

Something with binary encoding. Investigating.

Test case in att/get.js

Nowhere did you mention that the callback format is

(err, body, headers)

You also don't mention whether body is a string or whether you json parsed it internally.

Oh and then your examples are (err, response, headers) ¬_¬

Turns out from the source that the second argument is a JSON parsed body if it's not binary.

Convenience functions for attachments

The way it is implemented gives inconsistent results in the moment auth is called with one of the arguments empty and when both arguments are filled in but user can't be authenticated. In the first case we get back SUCCESS and in the second we get proper result with ERROR with username or password incorrect message. In my opinion this should return always error. The issue here is this line in relax: if(cfg.user && cfg.pass)

Additionally if we have auth we also should have corresponding session deleting function (unauth?). Even if it is very easy to achieve with relax it would be more consistent to have it separately or maybe use auth with empty params instead.

if you agree on this I can do necessary changes.

cheers,
Artur

Okay, so I'm using couchdb + node.js to make a bot in a pet capacity.
i.e., you an use text commands to level it up, teach it tricks, and so on and so forth.
Or at least that's the idea, as I can't seem to get couchdb to work. I'm not a big
fan of the RAM useage of MySQL, and mongo ticks me off to no end, but couch
seems relatively easy.

I've tried felix-couchdb, and cradle, but nano works best for me.

Basically, through calling other functions, vars change. Primarily 'exp' and 'level'.
However, when these change, and mSave(); (the function to insert, see below)
is called, the changes aren't noted. Let's say it saves with Level:2 and Exp:50.
When I restart the node process, and it gets the information, it has the default value of 0.

global.mDBName = 'pet2';
global.mHunger = 100;
global.mExp = 0;
global.mLevel = 0;
global.mName = 'BotOne';
global.mType = 1;
global.mUserId = '4fb1c582aaa5cd1b4900001e';

Log("Connecting to couchdb");
global.nano = require('nano')('http://localhost:5984');

Log("Finding database");
nano.db.create(mDBName, function(a) { a ? Log("db found, connecting") : Log("db not found, creating")
});

global.mSave = function() {
  store.insert({name:mName, type:mType, exp:mExp, hunger:mHunger, level:mLevel}, mUserId, function(a, b) {
    a || console.log(a)
  })
  Log('Bot Saved')
};

store.get(mUserId, function(b, a) {
    if(b && "not_found" == b.error) {
      mSave(), Log("Doc not found, creating")
    }else {
      if(b) {
        return console.log(b)
      }
      Log("Connected to doc: name:" + a.name + ", type:" + a.type + ", level:" + a.level + ", exp:" + a.exp + ", hunger:" + a.hunger);
      mHunger = a.hunger;
      mExp = a.exp;
      mLevel = a.level
    }
  });

I need to use _all_docs with the 'include_docs: true' parameter in order to fetch multiple documents in one call. I've tested in my app and I'm getting an immense performance improvement (e.g. from 5 mins down to ~15 seconds) when retrieving documents in batches of 100 at a time from a set of ~3000 documents. I cannot do this in nano today because bulk_docs does not take params. I see two potential solutions: minimal impact but breaking API change or add a new function.

Minimal: function bulk_docs(docs,callback) -> function bulk_docs(docs, params, callback)

New Function: function get_docs(doc_names, params, callback)

thoughts?

Namely:

• _uuids
• _stats
• _config
• _active_tasks
• _all_docs_by_seq

Doesn't work if I have GNU nano installed.
http://www.nano-editor.org/

I have struggled with this myself when using request directly. Often I want to do this:

do_it(function(error, body, header) {
    if(error)
        throw error;

    console.log("Everything is fine");
    // Do more stuff now.
})

Unfortunately, it is better to throw a new Error() instead of a plain object. This provides traceback information. So I must do this:

do_it(function(error, body, header) {
    if(error)
        throw new Error("Couch error: " + JSON.stringify(error));

    console.log("Everything is fine");
    // Do more stuff now.
})

But what if there was a socket error and I already have an Error object? It is all a bit messy.

What do you think about this (I have not tried it in code, just thinking about it now). When Couch returns an error:

1. Create a new Error with perhaps a helpful description of what went wrong (e.g. "Failed to create document")
2. Copy the CouchDB error object on top of the Javascript error object
3. Return that in the callback

This way, the user has a choice:

do_it(tries, function on_done(error, body, header) {
    if(error.message == "no_db_file")
        return do_it(tries+1, on_done);
    else if(error)
        throw error;

    console.log("Everything is fine");
    // Do more stuff now.
})

Would that work or is there a problem with the idea? Thanks!

When no callback is passed to relax, the request is sent before the body field is set. When i do a db.insert(doc) without the callback, the doc will not show up in the db, because no data is sent. Is this intended?

Instead of calling iris couch.

This would make travis-ci integration easier and tests would run much faster

Scan code to make sure no exceptions can be raised by request.

Hi, Nuno.

Is it appropriate to integrate any Follow functionality into Nano?

On the one hand, Follow also strives for simplicity and tries to mirror the request API. The other major advantage IMO is the inactivity timeout which is the only way to detect some kinds of error.

On the other hand, I am not sure if it is "feature bloat" or "mission creep" for Nano.

Do you have any thoughts about this?

I've read through the documentation, and I may be doing this wrong, but when I try to insert an image, from file, into a currently existing document and I do:

      fs.createReadStream(doc.image.path).pipe(
                      db.attachment.insert(doc._id, doc.image.name, {}, doc.image.mime)                        
     );

I get back a 409 from couchdb, am I implementing this incorrectly?

Sorry to keep firing issues at you - don't worry, I'll do a pull request for this if you agree its a problem. :)

request uses a global cookie jar, into which all response cookies are stored, and then used on subsequent requests. This behaviour is enabled by default; you have to set jar to false to disable it.

I believe that nano should always default for jar to false.

Because of the nature of nano and CouchDB, it is likely that you will doing multiple nano requests as different users (especially if you have a 1 user to 1 database set up) within a single method.

This probably hasn't been noticed before as nano hasn't had support for CouchDB's cookie authentication. But with this coming (#71), it is especially problematic having jar enabled by default - it means that if you have performed an action as a logged in standard user (cookie auth), and then attempt something as admin (using basic auth) - the admin request will fail - because the cookie jar sends over the auth cookie of the standard user - irrespective of whether you have isolated nano instances (because request's cookie jar is global).

See this gist for an example: https://gist.github.com/2717116

It should just be a case of changing this code from:

if (opts.jar) { 
  req.jar = opts.jar;
}

to

req.jar = opts.jar || false;

The interim workaround is to do the above, by using request_defaults (as below), but as I see no benefit for nano in using a cookie jar in any scenario, I think the fix above should be baked in instead.

var nano = require('nano')({ url: 'http://localhost:5984', request_defaults: { jar: false }});

Hi, I'm new to couch, so I may miss something. I'm looking for a way to update an existing document but can't find that in the API, is this something I'm supposed to write myself?

Just throwing an idea out there for consideration...

What if the order of the callback parameters was err, body, headers? IMHO opinion, the headers are a little less often needed than the body, aren't they? If the positions were changed, writing a callback that uses the body but not the headers would be function (err, body) instead of function(err, _, body). I find the former to be a lot more pleasant to read.

I realise that this change breaks backward compatibility, but that shouldn't be too big of an issue since we didn't hit 1.0 yet.

Anyway, maybe it's just me. Maybe my use case isn't shared with many others and this entry may be used as some kind of survey.

Also can't get verbose output

it would be nice if they were buffers :)

That should help making them more readable and hopefully help people contributing

Hello,

I would like to implement support for CouchDb Document Update Handlers.

http://wiki.apache.org/couchdb/Document_Update_Handlers

My thoughts are implementing a new function called update_doc:

   /*
    * calls document update handler
    *
    * @param {design_name:string} design document namd
    * @param {update_name:string} update method to call
    * @param {doc_name:string} document name to update
    * @param {params:object} additions to the querystring
   */   
  function update_doc(design_name, update_name, doc_name, params, callback) {
     if(typeof params === "function") {
       callback = params;
       params = {};
     }
     var update_path = '_design/' + design_name + '/_update/' + update_name + '/' + doc_name;
     return relax({db: db_name, path: update_path, method: "PUT", params: params}, callback);
   }

Then a public db alias called "update"

  public_functions.update = update_doc;

usage

This should provide the ability to easily execute atomic updates on the couchdb server:

Document Update Handler in Design Document called my_design_doc:

"updates": {
  "in-place" : "function(doc, req) {
      var field = req.query.field;
      var value = req.query.value;
      var message = 'set '+field+' to '+value;
      doc[field] = value;
      return [doc, message];
  }"
}

Then using nano:

@db.update("my_design_doc", "in-place", "<doc_name>", { field: "foo", value: "bar" }, function(e,b) { console.log(b); }); 

Thoughts?

Verbose mode can be incredibly useful for debugging even in production, instead of just throwing console.log all over the place, we could allow the caller to specify their own custom function for capturing the logs, if a user provides a transport, always pipe to it, otherwise if you're in testing mode, just have that function be console.log

Thoughts?

Related to #29

Nano sometimes interpolates user input without proper escaping.

Should we concern ourselves with protecting from URL injection? Or should the library user take care of that? Is this even a problem?

Hi, Nuno. Is it correct to include node_modules in the Git code? I thought that is out of scope for source builds (obviously it is put there by npm install but that is a "binary build" IMO).

In my own code I always put node_modules in the .gitignore however I have no idea if that is correct or not.

Any thoughts?

We're currently building an app for which we use a 1 database to 1 user architecture. Along with this we're using CouchDB's cookie authentication mechanism.

I'd like to submit a pull request that will allow the user to supply cookie as part of params. If this is set then the two required headers (X-CouchDB-WWW-Authenticate and cookie) will be added to the request in the relax function.

As per the contribute section of the README, I'm submitting this here first to get views/acceptance on the proposal before submitting a pull request.

Sample usage:

alice.get('rabbit', { revs_info: true, cookie: req.cookies['cookiename'] }, function(err, body) {
   // Pass-through cookie if couchdb has generated a new value
   if (headers && headers['set-cookie'])
     res.cookie('cookiename', headers['set-cookie'], { httpOnly: true });
   if (!err)
     console.log(body);
});

HEAD requests have empty bodies, seen below:

>>
{ method: 'HEAD',
  headers: 
   { 'content-type': 'application/json',
     accept: 'application/json' },
  uri: 'http://admin:admin@localhost:5984/tests-master-profile-mappings/pm-d373c6bf6346cd8da895d3ec0f73cc92' }
<<
{ err: 'couch',
  body: undefined,
  headers: 
   { date: 'Mon, 02 Jan 2012 23:30:34 GMT',
     'content-type': 'application/json',
     'cache-control': 'must-revalidate',
     'status-code': 404 } }

I receive the following error:

   uncaught undefined: TypeError: Cannot read property 'reason' of undefined
    at Request._callback (/node_modules/nano/nano.js:167:38)
    at Request.callback (/node_modules/nano/node_modules/request/main.js:99:22)
    at Request.<anonymous> (/node_modules/nano/node_modules/request/main.js:361:18)
    at Request.emit (events.js:64:17)
    at IncomingMessage.<anonymous> (/node_modules/nano/node_modules/request/main.js:327:16)
    at IncomingMessage.emit (events.js:81:20)
    at HTTPParser.onMessageComplete (http.js:133:23)
    at Socket.ondata (http.js:1231:22)
    at Socket._onReadable (net.js:683:27)
    at IOWatcher.onReadable [as callback] (net.js:177:10)

I believe this code is trying to signal an error, however, in the case of a HEAD request, this should not be relying on the existence of a body, since by definition, a HEAD request will only return headers in CouchDB. Thoughts?

Namely, Membase

When I am trying to pass an array as an argument with updateWithHandler the couchdb update function is only being passed the last argument of the array.

For example when I do this:

db.updateWithHandler('test','take_array', "some_id", {"my_array": ["abc","123"]}, function(){ ... })

in the update function req.query.my_array == "123"

I'm guessing there is an issue when converting the JSON object of arguments into a query string

Find a way to write tests for both browser and nodejs

Make tests run in Travis

http://wiki.apache.org/couchdb/Replication#Cancelling_a_continuous_replication_task

rather than explicitly supporting continuous as the only option, what we really need is a way to pass any options to the request body, cancel, continuous, doc_ids, are examples of legitimate attributes for replication requests. also, filters and query_params for filtered replication, we need support for all of that.

I'm happy to contribute this, not sure what your opinions are on design/backwards compatibility.

Package.json has errs as dev dependency instead of dependency. Causes the following error on require('nano'):

require('nano')
Error: Cannot find module 'errs'
at Function._resolveFilename (module.js:332:11)
at Function._load (module.js:279:25)
at Module.require (module.js:354:17)
at require (module.js:370:17)
at Object. (/Volumes/MacHD2/verticals/cs-whereis-poi-extract/node_modules/nano/nano.js:21:19)
at Module._compile (module.js:441:26)
at Object..js (module.js:459:10)
at Module.load (module.js:348:31)
at Function._load (module.js:308:12)
at Module.require (module.js:354:17)

In nano 1.3.2, the port number gets junked:

nano('http://127.0.0.1:5984/db')  =>  config: { url: 'http://127.0.0.1', db: 'db' }

A simple rewrite would be:

cfg.url = u.format({protocol:path.protocol,host:path.host})

While I'm at it I'd like to suggest to do

db = path_array.pop()
cfg.url = u.format({protocol:path.protocol,host:path.host,pathname:path_array.join('/')})

so that the path doesn't get junked as well:

nano('http://127.0.0.1:5984/path/to/db')  =>    config: { url: 'http://127.0.0.1:5984/path/to', db: 'db' }

@jchris identified this as a problem and solved it in coux. We should support this too.