hannahwhy / analysand

a CouchDB client library of dubious worth (mirror)

analysand's Issues

ChangeWatcher's connection-checking mechanism needs to be able to use credentials

A CouchDB database that has a nonempty member list will restrict access to its change feeds. ChangeWatcher needs to be able to deal with this scenario.

At present, ChangeWatcher treats 4xx errors as connection errors. That's fine, but we should be able to apply the same request-customization code to connection checking.

Stream views

CouchDB views may contain a great many documents. At present, Analysand parses the entire JSON stream and only presents the completed object. This won't work for e.g. _all_docs or large view ranges.

I'd like to write it like this:

resp = db.view('video/by_editor_and_title', :startkey => ['foo', 'bar'], :endkey => ['baz', 'xyz']) do |view, row|
  view.total_rows # total number of rows
  view.offset # => the offset
  row # => { "id": "doc_id", "key": ["foo", "bar"], "value": { ... } }, etc.  basically, the current row
  break # to abort streaming
end

resp.success? # => true or false
resp.offset # => the offset
resp.total_rows # => total number of rows
resp.rows # raises NoMethodError: a streaming response doesn't respond to this; rows aren't kept around

db.all_docs do |view, row|
  # same idea
end

YAJL can be used for this -- it is capable of streaming individual elements in a JSON array -- but the yajl-ruby gem, as far as I can tell, buffers incoming data until a complete JSON object is constructed. (It's possible to stream multiple JSON objects over the wire, which is what Analysand::ChangeWatcher needs to work with, but not to receive e.g. rows in an array as they're built.)

Analysand::Http::_req does not escape document IDs clearly

I'm running into an issue where a partner's database has keys that look like 10067--http://ark.cdlib.org/ark:/13030/kt067nc38g. The docs for Analysand::Database note the following:

  # Note: CouchDB treats forward slashes (/) specially.  For document IDs, /
  # denotes a separator between document ID and the name of an attachment.
  # This library makes use of that to implement attachment storage and
  # retrieval (see below).
  #
  # If you are using forward slashes in document IDs, you MUST encode them
  # (i.e. replace / with %2F).

In my case, I've made sure that I've escaped the IDs (using CGI.escape), before they even get passed to Analysand::Http::_req. This ensures an ID like 10067--http%3A%2F%2Fark.cdlib.org%2Fark%3A%2F13030%2Fkt067nc38g gets passed to Analysand::Http::_req.

However, given that % is a reserved character in URIs, it gets escaped again (when Analysand::Http::_req calls URI.escape, which means the actual document ID that gets requested is 10067--http%253A%252F%252Fark.cdlib.org%252Fark%253A%252F13030%252Fkt067nc38g. (In other words, the % characters get double-escaped.)

ChangeWatcher does silly things re: use of nonblocking I/O

https://github.com/yipdw/analysand/blob/master/lib/analysand/change_watcher.rb#L161-164

The above can be replaced with

while @running
  @http_parser << @socket.read(QUANTUM)
end

EOFError when connecting to Iris Couch-hosted databases

When connecting to Iris Couch-hosted databases, all operations (yeah, even #ping) are closed with EOFError. Figure out why.

Make it easier to use cookies from CouchDB requests to prolong sessions

When using CouchDB with session cookies, renewed session cookies may be issued by CouchDB if the supplied session cookie meets a number of criteria.

One such criterion in CouchDB 1.2.0: A new cookie will be issued if the supplied cookie is not expired and was issued within 10% of the timeout window. Example: if the timeout window is 600 seconds, and the cookie was issued later than .1 * 600 = 60 seconds ago, a new cookie will be issued.

Analysand's present methods return a response from CouchDB, but they do not yet provide any way of accessing a renewed session cookie. Add one.