GithubHelp home page GithubHelp logo

couchbase / couchbase-lite-ios Goto Github PK

View Code? Open in Web Editor NEW
1.6K 84.0 295.0 65.95 MB

Lightweight, embedded, syncable NoSQL database engine for iOS and MacOS apps.

License: Apache License 2.0

Objective-C 52.07% C 0.47% Shell 0.83% Objective-C++ 10.44% Ruby 0.06% Swift 33.14% C++ 0.11% JavaScript 2.23% SCSS 0.42% Mustache 0.24%
database nosql sync mobile swift objective-c ios macos

couchbase-lite-ios's Introduction

Couchbase Lite for iOS and MacOS

License Build Status Coverage Status

Couchbase Lite is an embedded lightweight, document-oriented (NoSQL), syncable database engine.

Couchbase Lite implementation is on top of Couchbase Lite Core, which is also a new cross-platform implementation of database CRUD and query features, as well as document versioning.

Requirements

  • iOS 12.0+ | macOS 12.0+

Installation

Swift Package

Requirements:

  • XCode 12+
Community Edition
dependencies: [
        .package(name: "CouchbaseLiteSwift",
                 url: "https://github.com/couchbase/couchbase-lite-ios.git", 
                 from: "3.2.0-beta.1"),
    ],
Enterprise Edition
dependencies: [
        .package(name: "CouchbaseLiteSwift",
                 url: "https://github.com/couchbase/couchbase-lite-swift-ee.git", 
                 from: "3.2.0-beta.1"),
    ],

More detailed information on how to setup is available here: swift package manager

CocoaPods

You can use CocoaPods to install CouchbaseLite for Objective-C API or CouchbaseLiteSwift for Swift API by adding it in your Podfile:

Objective-C

Community Edition
target '<your target name>' do
  use_frameworks!
  pod 'CouchbaseLite'
end
Enterprise Edition
target '<your target name>' do
  use_frameworks!
  pod 'CouchbaseLite-Enterprise'
end

Swift

Community Edition
target '<your target name>' do
  use_frameworks!
  pod 'CouchbaseLite-Swift'
end
Enterprise Edition
target '<your target name>' do
  use_frameworks!
  pod 'CouchbaseLite-Swift-Enterprise'
end

Carthage

You can use Carthage to install CouchbaseLite by adding it in your Cartfile:

Community Edition
binary "https://packages.couchbase.com/releases/couchbase-lite-ios/carthage/CouchbaseLite-Community.json"
Enterprise Edition
binary "https://packages.couchbase.com/releases/couchbase-lite-ios/carthage/CouchbaseLite-Enterprise.json"

When running carthage update or build, Carthage will build both CouchbaseLite and CouchbaseLiteSwift framework.

How to build the framework files.

  1. Clone the repo and update submodules
$ git clone https://github.com/couchbase/couchbase-lite-ios.git
$ cd couchbase-lite-ios
$ git submodule update --init --recursive
  1. If not already installed, install doxygen, brew install doxygen

  2. Run ./Scripts/build_framework.sh to build a platform framework which could be either an Objective-C or a Swift framework. The supported platforms include iOS, tvOS, and macOS.

$ ./Scripts/build_framework.sh -s "CBL ObjC" -p iOS -o output    // For building the ObjC framework for iOS
$ ./Scripts/build_framework.sh -s "CBL Swift" -p iOS -o output   // For building the Swift framework for iOS

Documentation

Sample Apps

  • Todo : Objective-C and Swift

License

Like all Couchbase source code, this is released under the Apache 2 license.

couchbase-lite-ios's People

Contributors

borrrden avatar cflorion avatar chbeer avatar combinatorial avatar gitter-badger avatar jamesnocentini avatar jayahariv avatar jchris avatar keynuker avatar lambmj avatar lordkev avatar mastohhh avatar mglasgow avatar mhocouchbase avatar monowerker avatar mz2 avatar notjosh avatar pasin avatar pegli avatar rajagp avatar refractalize avatar robertjpayne avatar sethrosetter avatar snej avatar tfmd avatar tleyden avatar tspacek avatar ufosky avatar velicuvlad avatar zgramana avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

couchbase-lite-ios's Issues

Pull replication between two local DB instances fails

I have added the CBL listener framework to a CBL project and I have set up a pull replication between two local DB's, the replication appears to start OK but fails with a connection error after a few seconds, no documents are sync'd.

A push replication between the same two local DB's works correctly and documents are sync'd.

I have included the console log below from the pull sync.

12:21:04.410| Sync: ReplicatorManager: Validating CBLRevision[3BAD..724A/(null)]: {
    continuous = 1;
    source = "http://server1.cblite./sync_gateway/";
    target = "mydb";
}
12:21:04.415| Sync: ReplicatorManager: {3BADAF85-E9B2-4B75-8B09-757BA0E9724A #1-9d88645e0c2304f22e222da09f5823c7} was created
12:21:04.417| Sync: CBL_Puller[http://server1.cblite./sync_gateway/] STARTING ...
12:21:04.418| CBLBasicAuthorizer initWith <http://server1.cblite./sync_gateway/>
12:21:04.423| Sync: CBL_Puller[http://server1.cblite./sync_gateway/]: Going online
12:21:04.448| Sync: ReplicatorManager: Updating {3BADAF85-E9B2-4B75-8B09-757BA0E9724A #1-9d88645e0c2304f22e222da09f5823c7} with {
    "_replication_id" = "A6A8E0D5-92A4-4B79-B869-8155DC27DA83";
    "_replication_state" = triggered;
    "_replication_state_time" = 1360326064;
}

12:21:05.694| Sync: ReplicatorManager: Validating CBLRevision[3BAD..724A/(null)]: {
    "_id" = "3BADAF85-E9B2-4B75-8B09-757BA0E9724A";
    "_replication_id" = "A6A8E0D5-92A4-4B79-B869-8155DC27DA83";
    "_replication_state_time" = 1360326064;
    "_rev" = "2-7d32d9ef58b9bd2a839bcedd2de03549";
    continuous = 1;
    source = "http://server1.cblite./sync_gateway/";
    target = "mydb";
}
12:21:05.717| Sync: ReplicatorManager: Restarting replicator for {3BADAF85-E9B2-4B75-8B09-757BA0E9724A #3-df105799cb09668b379549144073948e}
12:21:05.718| Sync: CBL_Puller[http://server1.cblite./sync_gateway/] STOPPING...
12:21:05.720| Sync: Stopping 1 remote requests
12:21:05.723| Sync: CBL_Puller[http://server1.cblite./sync_gateway/] STOPPED
12:21:05.724| Replication: CBL_Puller[http://server1.cblite./sync_gateway/] took 1.301 sec; error=(null)
12:21:05.727| Sync: ReplicatorManager: Updating {3BADAF85-E9B2-4B75-8B09-757BA0E9724A #3-df105799cb09668b379549144073948e} with {
    "_replication_id" = "A6A8E0D5-92A4-4B79-B869-8155DC27DA83";
    "_replication_state" = completed;
    "_replication_state_time" = 1360326065;
}
12:21:05.752| Sync: ReplicatorManager: {3BADAF85-E9B2-4B75-8B09-757BA0E9724A #3-df105799cb09668b379549144073948e} was created
12:21:05.754| Sync: CBL_Puller[http://server1.cblite./sync_gateway/] STARTING ...
12:21:05.755| CBLBasicAuthorizer initWith <http://server1.cblite./sync_gateway/>
12:21:05.760| Sync: CBL_Puller[http://server1.cblite./sync_gateway/]: Going online
12:21:05.765| Sync: ReplicatorManager: Updating {3BADAF85-E9B2-4B75-8B09-757BA0E9724A #3-df105799cb09668b379549144073948e} with {
    "_replication_id" = "A6A8E0D5-92A4-4B79-B869-8155DC27DA83";
    "_replication_state" = triggered;
    "_replication_state_time" = 1360326065;
}
12:21:05.970| Sync: CBL_Puller[http://server1.cblite./sync_gateway/]: Replicating from lastSequence=(null)
12:21:06.100| Sync: ReplicatorManager: Updating {3BADAF85-E9B2-4B75-8B09-757BA0E9724A #5-a20a4604e88d93dbb79cced18510094c} with {
    "_replication_id" = "A6A8E0D5-92A4-4B79-B869-8155DC27DA83";
}

12:21:18.733| �WARNING*** : CBLSocketChangeTracker[0x1c5a25f0 sync_gateway]: Can't connect, giving up: Error Domain=kCFErrorDomainCFNetwork Code=2 "The operation couldn’t be completed. (kCFErrorDomainCFNetwork error 2.)" UserInfo=0x1c5a79e0 {kCFGetAddrInfoFailureKey=8}
12:21:18.742| Sync: CBL_Puller[http://server1.cblite./sync_gateway/]: ChangeTracker stopped; error=Error Domain=kCFErrorDomainCFNetwork Code=2 "The operation couldn’t be completed. (kCFErrorDomainCFNetwork error 2.)" UserInfo=0x1c5a79e0 {kCFGetAddrInfoFailureKey=8}
12:21:18.746| Sync: ReplicatorManager: Updating {3BADAF85-E9B2-4B75-8B09-757BA0E9724A #5-a20a4604e88d93dbb79cced18510094c} with {
    "_replication_id" = "A6A8E0D5-92A4-4B79-B869-8155DC27DA83";
}

No public API to access local documents

There's currently no way to access local documents in the new API.

Using a doc-ID prefix "_local/" doesn't work, because that mapping was provided by the REST implementation, not the CBLDatabase class.

Assertion Failure when stopping/starting replications

Hi,

We've recently updated (pulled) the latest Couchbase Lite (CBL) master branch, after having used the TouchDB public-api branch for the past few months. I'd like some help diagnosing a problem that we've seen before, but with this latest update, seems to have become more prevalent.

Our use case:
Our iOS (iPad) app allows users to create new projects and open existing projects. When creating or opening a project, the old project is closed. We maintain one database per project. When creating or opening a project (database), we begin continuous bi-directional replication. When a different project (database) is created or opened, we stop replication with the old database, and start replication with the new database. To make matters more complicated, we also have in-app scenarios where we pause (stop) replication temporarily, and resume (restart) later with the same database. We do NOT use persistent replications. We manually manage when our replications start and stop.

The problem:
We hit the following assertion failure quite often when stopping our replication with a Cloudant server, and then trying to start up replication with a different database.

2013-03-18 13:02:52.199 Hammersmith_KIFTests[28082:907] *** ASSERTION FAILED: Assertion failed: Unexpected value for tdReplicator: CBL_Puller[https://myacct:[email protected]/proj_afe9f595-e8ed-4207-9851-ca70121ba241] (expected CBL_Puller[https://myacct:[email protected]/proj_afe9f595-e8ed-4207-9851-ca70121ba241])
2013-03-18 13:02:52.201 Hammersmith_KIFTests[28082:907] *** Assertion failure in -[CBLReplication bg_replicationProgressChanged:], /Users/toddor/hs-ios/Hammersmith/external-libs/cbl/Source/API/CBLReplication.m:435

Backtrace wasn't terribly useful, but I can provide that, if needed. Also, I've modified the user name and credentials in the URL to keep them private.

The assertion failure seems to have become a bigger problem since we updated to the latest CBL. And the problem seems to be that the replication class thinks its current replicator is different than it should be. However, looking at the failure, the two URLs listed appear identical. More importantly, this assertion failure seems very dependent on the timing of our stopping and starting replications. However, we do have scenarios where this is very simple to reproduce.

Any help would be greatly appreciated.

Thanks,
-Todd

request for _changes using internalURL gives kCFHostErrorUnknown in change tracker

I'm trying to set up local replication between two databases, and it seems like the NSURLProtocol isn't being picked up by CBLSocketChangeTracker. Here's the test code:

self.sourceDb = [[CBLManager sharedInstance] createDatabaseNamed:kSourceDbName error:nil];
for (unsigned i=0; i<20; i++) {
    NSDictionary * properties = @{@"name":@"test",@"sequence": @(i)};
    CBLDocument * doc = [self.sourceDb untitledDocument];
    [doc putProperties: properties error:nil];
}
self.targetDb = [[CBLManager sharedInstance] createDatabaseNamed:kTargetDbName error:nil];
self.replication = [self.targetDb pullFromURL:self.sourceDb.internalURL];
[self.replication addObserver:self forKeyPath:@"completed" options:0 context:nil];
[self.replication addObserver:self forKeyPath:@"mode" options:0 context:nil];
[self.replication start];

The log shows that the initial connection to http://lite.couchbase./source/_local/235aa4ef8704428da3bcd63596f7efc7269d840f works fine but the change tracker requests for //lite.couchbase./source/_changes are failing with a kCFHostErrorUnknown error with kCFGetAddrInfoFailureKey of 8 ("hostname nor servname provided, or not known"). I put in some debugging and saw that CBL_URLProtocol canInitWithRequest: was not being called for HTTP requests originating in the CBLSocketChangeTracker class, probably because the change tracker is using CFHTTPMessageCreateRequest instead of NSURLRequest.

Full debug log is here: https://gist.github.com/pegli/5447998

Add queue method to CBLDatabase to run Asynchronous workloads

Currently, you need to create a queue or a thread of your own to run CBL workloads in the background. CBL_Server has a queue method which allows you to perform asynchronous workloads. To solve this problem and keep CBL_Server private, add a queue method to CBLDatabase which then accesses it's private manager and server to queue the block of work.

Latest CBL accessing /_session

I have an issue I can't get to the bottom of.

I have CBL/sync_gateway working fine, and I have Persona logon working fine, but when I try to setup CB:/sync_gateway to use Person Authentication CBL seems to be trying to access the /_session without the DB prefix.

I have updated to the latest CBL/sync_gateway 14th April, so I think this is a setup issue but I can't figure out what is causing the behaviour.

Here are some details of my setup config:

sync_gateway

In my config I have added the following to force authentication:

"users": {
"GUEST": {"disabled": true}
},

I have the following Logging enabled:

"log": ["CRUD", "REST", "REST+"],

On the sync_gateway command line I have added:

-personaOrigin "http://Macintosh-40.local:4984/"

On startup the only info in the console relating to authentication is:

Reset guest user to config

On the Client side

I initially setup a replication without authentication and I see the following in the sync_gateway console:

10:30:18.124159 Auth failed for username="", cookie=
10:30:18.124200 HTTP: --> 401 Invalid login
10:30:18.124978 Auth failed for username="", cookie=
10:30:18.125013 HTTP: --> 401 Invalid login
10:30:18.175147 HTTP: GET /mydb/_local/6a3ee783b4eb34bcba1568d3834010cc15eaf670
10:30:18.175742 Auth failed for username="", cookie=
10:30:18.175783 HTTP: --> 401 Invalid login
10:30:18.214173 HTTP: GET /mydb/_local/c0f0d4a7564c4d98e5a3d83d68d4dbc30a6a0b26
10:30:18.214554 Auth failed for username="", cookie=
10:30:18.214599 HTTP: --> 401 Invalid login

Which seems reasonable

I call the Persona login with

startBrowserIDWithOrigin:@"http://Macintosh-40.local:4984/"

Once the user logs in with Persona I setup a new db and replication with this code:

    NSArray* replsModel = [_database replicateWithURL:_syncGatewayURL exclusively: YES];


    CBLReplication *pullRepl = (CBLReplication *)[replsModel objectAtIndex:0];
    bool pullAsserted = [pullRepl registerPersonaAssertion: _assertion];

    pullRepl.filter = @"sync_gateway/bychannel";
    pullRepl.query_params = [NSDictionary dictionaryWithObjectsAndKeys:@"*",@"channels", nil];
    pullRepl.persistent = YES;
    pullRepl.continuous = YES;

    CBLReplication *pushRepl = (CBLReplication *)[replsModel objectAtIndex:1];
    bool pushAsserted = [pushRepl registerPersonaAssertion: _assertion.rawAssertion];

    pushRepl.persistent = YES;
    pushRepl.continuous = YES;

Then I see this in the client console

10:02:49.370| Sync: ReplicatorManager: Validating CBLRevision[D9AE..CEA1/]: {
"_id" = "D9AE6C7F-D4FF-4A0D-8C61-4C3669C1CEA1";
"_replication_id" = "B1E4A486-6759-4D76-B22D-C039EA1F1DD6";
"_replication_state_time" = 1365984469;
"_rev" = "5-cb9e1ddad583ab03e85e9f3445009f1f";
continuous = 1;
filter = "sync_gateway/bychannel";
"query_params" = {
channels = "*";
};
source = {
auth = {
persona = {
email = "[email protected]";
};
};
url = "http://Macintosh-40.local:4984/mydb";
};
target = "mylocaldb";
}
.
.
.
10:02:49.594| Sync: ReplicatorManager: Validating CBLRevision[2CD7..C538/]: {
"_id" = "2CD77F55-489A-4272-94D5-9E43FE1CC538";
"_replication_id" = "EC89B67F-D896-4C73-B6B7-B62F47D6350A";
"_replication_state_time" = 1365984469;
"_rev" = "5-5fd8c75769374a687925eedbef454cf5";
continuous = 1;
source = "mylocaldb";
target = {
auth = {
persona = {
email = "[email protected]";
};
};
url = "http://Macintosh-40.local:4984/mydb";
};
}
.
.
.
10:02:49.868| CBLRemoteJSONRequest[GET http://Macintosh-40.local:4984/_session]: Got error Error Domain=CBLHTTP Code=404 "404 not_found" UserInfo=0x1cd60ca0 {NSURL=http://Macintosh-40.local:4984/_session, NSLocalizedFailureReason=not_found, NSLocalizedDescription=404 not_found}
10:02:49.869| Sync: CBL_Puller[http://Macintosh-40.local:4984/mydb]: Session check failed: Error Domain=CBLHTTP Code=404 "404 not_found" UserInfo=0x1cd60ca0 {NSURL=http://Macintosh-40.local:4984/_session, NSLocalizedFailureReason=not_found, NSLocalizedDescription=404 not_found}
10:02:49.869| Sync: CBL_Puller[http://Macintosh-40.local:4984/mudb]: postProgressChanged (0/0, active=1 (batch=0, net=1), online=1)
10:02:49.870| Sync: CBL_Puller[http://Macintosh-40.local:4984/mydb]: postProgressChanged (0/0, active=0 (batch=0, net=0), online=1)
10:02:49.870| CBLRemoteJSONRequest[GET http://Macintosh-40.local:4984/_session]: Got error Error Domain=CBLHTTP Code=404 "404 not_found" UserInfo=0x1cddf8a0 {NSURL=http://Macintosh-40.local:4984/_session, NSLocalizedFailureReason=not_found, NSLocalizedDescription=404 not_found}
10:02:49.871| Sync: CBL_Pusher[http://Macintosh-40.local:4984/mydb]: Session check failed: Error Domain=CBLHTTP Code=404 "404 not_found" UserInfo=0x1cddf8a0 {NSURL=http://Macintosh-40.local:4984/_session, NSLocalizedFailureReason=not_found, NSLocalizedDescription=404 not_found}
10:02:49.872| Sync: CBL_Pusher[http://Macintosh-40.local:4984/mydb]: postProgressChanged (0/0, active=1 (batch=0, net=1), online=1)
10:02:49.872| Sync: CBL_Pusher[http://Macintosh-40.local:4984/mydb]: postProgressChanged (0/0, active=0 (batch=0, net=0), online=1)

Nothing at all is written to the sync_gateway console

I think the problem is that /_session is being called instead of /mydb/_session, but also I was not sure if _persona should be called rather than _session.

I have checked the source and it is using the ..Persona.. version of the Authorizer class, I have rebuilt the framework twice so I think I have the latest CBL framework in my project.

I suspect I have some bad config somewhere, but I have failed to find it so far.

Attachments RAM issue

When pulling ~100 documents with large (~1-2 MB attachments), it seems CBL never releases that memory and RAM creeps up. Attached is a screenshot of the allocation trace for the objects 'still living'.

attachment

attachment bodyURL returning file path with generic extension '.blob'

Attachment data is stored as a file with the extension .blob
Using attachment bodyURL the file path to attachment data is obtained.
I want to play a mov attachment in AVPlayer but it expects a file with a know extension, e.g. mov, m4v.

As a work around I do the following

NSError *err = nil;
NSURL* url = [[self attachmentNamed:@"question"] bodyURL];
NSURL* other = [[url URLByDeletingPathExtension] URLByAppendingPathExtension:@"mov"];
[[NSFileManager defaultManager] linkItemAtURL:url toURL:other error:&err];
[[AVPlayer alloc] initWithURL:url];

Symbolic links do not work. I then need to tidy up these linked files at some point. I thought maybe when the app terminates but this isn't good if the app is long running and lots of attachments and created and deleted. Ideally attachment data would be stored in files using the original file extension if available when created. If implemented, is this a pull request you'd accept? Is it something that you are planning on doing? Or is it not needed?
If this isn't possible is there a callback, notification, or property I could observe to tell me the data is being updated/deleted and I should delete stale file links. Thanks for any help.

Deletes don't appear to propagate when replicating channels

I set up a test app with Couchbase Lite and sync-gateway and I while I can see my channel documents replicate properly between two devices, if I try to delete the same documents from one device the delete does not appear to replicate to the second device and it retains the document in its local database.

Database name

This is a tiny one :)
I found at least one occurrence where +isValidDatabaseName: isn't doing its job.
When passed a database name that begins with a digit, it will say that's a valid name. However, when trying to create a database with that name, CBL will fail.

TouchDB transition : CBLDatabase not inheritable anymore ?

Hi there,

I have an app using TouchDB.
I had inherited CouchDatabase to handle my custom needs :

  • custom inits (mainly views definitions)
  • creation of template documents
  • replications logic
  • singleton

All my code is based arround this singleton class.
But doing the transition to CouchbaseLite, I can't manage to inherit CBLDatabase anymore and keep the same behavior.
I tried several hacks to get the [CBLManager createDatabaseNamed:error:] logic in my singleton class, that is, outside the CBLManager class, but it's bad, right ? And I couldn't anyway, since many things are internal and locked within the implementation.

Would you have some advices other than changing my architecture ?
Or wouldn't it be nice to be get back CBLManager inheritance possible ?

Thanks,
Philippe

Creating DB with invalid name fails silently

Currently using an invalid name (it appears the CouchDB rules below are enforced) for a database fails silently, i.e. the TDDatabase returned is nil but the error property is not populated.

I'm using the following code to cteate the database, in my initial use case I used an email address for the DB name.

TDDatabase *myDb = [dbManager createDatabaseNamed:@"[email protected]" error:&error];

I think the expected behavior would be to return nil for the database and populate the error with a description of the supported naming format.

Naming and Addressing (from CouchDB wiki)
A database must be named with all lowercase letters (a-z), digits (0-9), or any of the _$()+-/ characters and must end with a slash in the URL. The name has to start with a lowercase letter (a-z).

Syncing between 2 local databases currently possible?

Hi,

Do you know if having 2 local databases (one sync'ed from Dropbox on the local filesystem and another 'reference' one also on the local filesystem) is a possible use-case for the CBLDatabase -push/pull API? Couldn't find out how to setup this with current API. Thank you!

CBL local replication change notifications

It looks like internal replications (possibly all replications) are not correctly firing change notifications on the target datbase when it is local.

Test scenario

Scenario

Local CBL DB (A)
Local CBL DB (B)

Remote Sync_gateway (C)

I have a local persistent/continuous PUSH replication setup between A ---> B

I have validated that this is working correctly.

I have a persistent/continuous PUSH replication setup between B ---> C

I have validated that writing a document directly to B is correctly replicated to C

However documents replicated from A ---> B are not automatically replicated to C, if I directly write a document to B after replication from A ---> B has completed, the directly written document (in B) is replicated to C but NOT the documents that were previously replicated from A ---> B.

If I background/foreground or restart the App then the replicated documents in (B) are picked up and replicated to (C) successfully.

I enabled -LogCBLDatabase YES and see the following in the console

21:46:05.326| CBLDatabase: Begin transaction (level 1)...
21:46:05.359| CBLDatabase: Commit transaction (level 1)
21:46:05.375| CBLDatabase: Posting 1 change notifications
2013-02-24 21:46:05.378 meyumeapp[21409:150f] 21409: CFNetwork internal error (0xc01a:/SourceCache/CFNetwork/CFNetwork-609/Connection/URLConnectionClient.cpp:2341)
21:46:05.398| Sync: ReplicatorManager: Updating {5471EC1A-35A1-44E6-81B4-39FDE851F694 #7-241139593a5fe641d3c244813552c280} with {
"_replication_id" = "EB288E11-28EE-43FB-A92A-4086AB108408";

So it looks like the replicator is calling the CBLDatabase+Internal postChangeNotifications selector, but the notifications are not being received.

In the above you will see a CFNetwork error, but I do not know where this is coming from, only that it appears each time there is a notification from the replicator.

I am also seeing an issue with the live queries, where it does not update for docs inserted by the replicator, this also appears to be a problem with the receipt of notifications on the associated database.

All databases, views, queries e.t.c are being created in the AppDelegate or in view controllers, so they should all be running on the main thread and runloop.

'Database is busy'

It seems the underlying FMDB database is sometimes accessed from multiple threads at the 'same time'. My app isn't using any concurrency, dispatch queues or threads, yet I'm running into these issues when using live queries during a pull from a remote sync gateway. Not sure if it's relevant, but most of the documents being pulled contain attachments.

The log below show two different threads are trying to use the FMDatabase.

2013-04-09 07:12:13.414 Puller[58132:c07] -[FMDatabase executeUpdate:error:withArgumentsInArray:orVAList:]:740 Database busy (/Volumes/Home/Library/Application Support/iPhone Simulator/6.1/Applications/FDD4EE76-786A-4B29-8FF2-0ED674F2B847/Library/Application Support/CouchbaseLite/user-b56e5c11a783b8a0078d91d25b00588f.touchdb)
2013-04-09 07:12:13.415 Puller[58132:c07] Database busy
07:12:13.416| �WARNING*** : CouchbaseLite: Failed to rebuild view 'files/byIndex': 590
07:12:13.416| �WARNING*** : Failed to update view index: 590
2013-04-09 07:12:13.417 Puller[58132:4107] -[FMDatabase executeUpdate:error:withArgumentsInArray:orVAList:]:740 Database busy (/Volumes/Home/Library/Application Support/iPhone Simulator/6.1/Applications/FDD4EE76-786A-4B29-8FF2-0ED674F2B847/Library/Application Support/CouchbaseLite/user-b56e5c11a783b8a0078d91d25b00588f.touchdb)
2013-04-09 07:12:13.417 Puller[58132:4107] Database busy
07:12:13.418‖ �WARNING*** : CouchbaseLite: Failed to rebuild view 'files/byIndex': 590
07:12:13.418‖ �WARNING*** : Failed to update view index: 590
07:12:13.424| �WARNING*** : <CBLLiveQuery: 0x9379870>: Error updating rows: 590

add check for nonexistent file in CBLRemoveFileIfExists

The call to CBLManager replaceDatabaseNamed:withDatabaseFile:withAttachments:error: will remove any existing attachments directory for the named database prior to copying the specified source attachments directory into Library/Application Support/CouchbaseLite. In the case where the named database did not exist before the call, the method correctly returns YES but also sets a value for the error parameter. This happens because [NSFileManager removeItemAtPath:error:] in CBLRemoveFileIfExists is called with a nonexistent directory. I know that I'm being nit-picky, but can we change CBLRemoveFileIfExists to check for file existence prior to the attempt to delete?

return ![fmgr fileExistsAtPath:path] || [fmgr removeItemAtPath: path error: outError] || ![fmgr fileExistsAtPath: path];

With this change, we won't get an error object back on a success result.

Allow replicator processing delay to be set explicitly

At the moment, a processing delay is set by a #define in CBL_Replicator.m and directly in CBL_Puller.m. It would be nice to expose this so it can be changed by the app developer. Changing the delay could result in a higher load on the server but developers can trade that for a snappier app. For example, in a chat app, there's little reason to impose an artificial 0.5-1.0 second delay before sending the most recent message.

Support for geospatial queries

Support for geospatial queries, along the line of CouchGeo, but for iOS. For example, the ability to query for locations that exist in a particular box.

MYUtilities submodule

Running git submodule update results in the following error.

couchbase-lite-ios (master) $ git submodule update
fatal: reference is not a tree: 1c378ca0745c7ca504a39c4620d6b5448e042d9f
Unable to checkout '1c378ca0745c7ca504a39c4620d6b5448e042d9f' in submodule path 'vendor/MYUtilities'

CBLModel to-many relationships (array-of-CBLModel properties)

CBLModel should support to-many relationships: a property whose value is a collection of other CBLModels. For example,

@property (strong) NSArray* members;    // array items are Person objects

The underlying document property is of course a JSON array of strings (doc IDs). The tricky part is that, since the property declaration doesn't specify what types of objects the array should contain, there needs to be some sort of external annotation that specifies that the elements should be internalized as CBLModels, not NSStrings.

There's also the question of what subclass of CBLModel to instantiate — the CBLModelFactory can be used for that, or alternatively the external annotation could specify a specific subclass (which would help impose type-safety.)

Pusher warning: "removePending: sequence %d not in set"

This happened when pushing to sync_gateway. It didn't cause any 'visible' issues. I don't have a lot of context for this, sorry.

10:55:16.171| WARNING*** : CBL_Pusher[http://e1b1c275376b67ba7ce277e65602118a:*****@MY_IP:4984/sync_gateway] removePending: sequence 268 not in set, for rev {F93DCD81-FFDF-469C-A0BE-4E38CD639F0A #1-0b241700af1a0bb0334683364b81812a}
10:55:16.171| WARNING*** : CBL_Pusher[http://e1b1c275376b67ba7ce277e65602118a:*****@MY_IP:4984/sync_gateway] removePending: sequence 269 not in set, for rev {23EBE09B-5601-4F1A-B23C-532592C79387 #1-9c1ac2e156b72860c522f59011704636}
10:55:16.172| WARNING*** : CBL_Pusher[http://e1b1c275376b67ba7ce277e65602118a:*****@MY_IP:4984/sync_gateway] removePending: sequence 270 not in set, for rev {49F2C6F6-D6A7-4388-B7A1-82DA3F9830E3 #1-fdb456603c1a0eef9122121c4aab6bbd}
10:55:16.172| WARNING*** : CBL_Pusher[http://e1b1c275376b67ba7ce277e65602118a:*****@MY_IP:4984/sync_gateway] removePending: sequence 272 not in set, for rev {171B5E9C-5E81-4453-910D-F48F09894F12 #2-fd01315845c4bad1dfd3e8f0af416ba2}

attachment bodyURL returning file path with generic extension '.blob'

Attachment data is stored as a file with the extension .blob
Using attachment bodyURL the file path to attachment data is obtained.
I want to play a mov attachment in AVPlayer but it expects a file with a know extension, e.g. mov, m4v.

As a work around I do the following

NSError err = nil;
NSURL
url = [[self attachmentNamed:@"question"] bodyURL];
NSURL* other = [[url URLByDeletingPathExtension] URLByAppendingPathExtension:@"mov"];
[[NSFileManager defaultManager] linkItemAtURL:url toURL:other error:&err];
[[AVPlayer alloc] initWithURL:url];

Symbolic links do not work. I then need to tidy up these linked files at some point. I thought maybe when the app terminates but this isn't good if the app is long running and lots of attachments and created and deleted. Ideally attachment data would be stored in files using the original file extension if available when created. If implemented, is this a pull request you'd accept? Is it something that you are planning on doing? Or is it not needed?
If this isn't possible is there a callback, notification, or property I could observe to tell me the data is being updated/deleted and I should delete stale file links. Thanks for any help.

Delegate API to encode attachments on-disk

There's been a desire expressed to encrypt attachment bodies on disk, for security. CBL could implement a delegate API that would let developers interpose a filter between the attachment handling code and the filesystem. The app's implementation of the delegate could then manage the key and do the encryption.

The methods this need to hook into are the following on CBL_BlobStore:

- (NSData*) blobForKey: (CBLBlobKey)key;
- (NSInputStream*) blobInputStreamForKey: (CBLBlobKey)key
                                  length: (UInt64*)outLength;
- (BOOL) storeBlob: (NSData*)blob
       creatingKey: (CBLBlobKey*)outKey;

and in CBL_BlobStoreWriter:

- (void) appendData: (NSData*)data;

There's a question of whether the blob's key should be based on the SHA-1 digest of the original or the encoded data. I think the original is a better choice. However this does mean that the encoding delegate API needs to apply to all database instances (or at least all within a particular root directory) since mixing encoded and unencoded attachment bodies in the same directory can result in collisions.

Replication should run in a background thread

CBLReplication shouldn't run the replication in the same instance of CBLManager (i.e. on the same thread/queue.) The problems are:

  1. Doing replication work on the client thread can make it less responsive
  2. If the client is using CBL on a background queue, that queue won't have a runloop, and the replicator's network code requires a runloop, so it'll fail.

There is commented-out code in CBLReplication.m to run the replication in the manager's background server. I need to re-enable that and get it working.

Fix for 'Persona 'origin' URL should not end with '/'' breaks test Case

The removal of the trailing slash has broken the TEPersonaAuthorizer test case in the CBLPersonaAuthorizer.m class.

The problem is the original Assertion contained a trailing slash and now fails to match when looking up the assertion as the generated string from the origin URL no longer has the slash appended when it is built.

I think that there are more issues with the generation of the storage key and retrieval key which will result in misses in the Assertion dictionary. For example in the originForSite selector it will append a default port based on the protocol if one is not specified. This will result in the odd situation where supplying the same URL to the replicator and Persona origin will result in a miss when looking up the assertion.

I am running with a local patch that passes the origin string through the same code as the siteURL when generating the registration key, this ensures that the two keys will be identical for the same input URL.

I implemented this by updating the registerAssertion selector:

+ (NSString*) registerAssertion: (NSString*)assertion {
    NSString* email, *origin;
    NSDate* exp;
    if (!parseAssertion(assertion, &email, &origin, &exp))
        return nil;
    NSURL* originURL;
    if(!(originURL = [NSURL URLWithString:origin]))
        return nil;

    id key = @[email, [[self originForSite: originURL] absoluteString]];
    @synchronized(self) {
        if (!sAssertions)
            sAssertions = [NSMutableDictionary dictionary];
        sAssertions[key] = assertion;
    }
    return email;
}

The only reason I have not raised a pull request is I'm not sure how best to create the test cases with the need for canned Assertions.

Re-enable public continuous builds

In the building couchbase lite wiki page here:
https://github.com/couchbase/couchbase-lite-ios/wiki/Building-Couchbase-Lite
it says "Our build-bot runs hourly, 24/7/365, and generates builds of the latest commits." which links to here:
http://files.couchbase.com/developer-previews/mobile/ios/CouchbaseLite/

... which shows builds from February thru the end of March, but none after.

Not clear if the robot stopped or if the locations changed or if the wiki is out of date.

The project's README also points to this directory where it says: "(You might prefer to just download the latest build. But if you want to build it yourself...)"

So, my guess is the robot died.

Unparseable JSON data

During a pull replication from sync_gateway, CBL logged this warning about unparseable JSON data being received, along with the megabyte-long print of the data. The data is a JPG image attachment to a document.

It seems only part of the data was received during the pull, since the end } of the JSON dictionary isn't in the log.

WARNING*** : <CBLMultipartDocumentReader: 0x1c50ebc0>: received unparseable JSON data '{"_attachments":{"image":{"content_type":"image/jpeg","data":"/9j/4AAQSkZJRgABAQAAAQABAAD.....<<full image data here>>....gxlX2Ri/Aj9iPwTrmh2/izx5ap4y8Q6qn2u51DVZ99vNLu+8ifdSqHxC+KmtWfxR1Twn+z54L1Tx1rOmyRJEtjKlvp1nL/AH7m5f5E2f3Pv1lfst+OvFf7T+lap4K/srW9L+FDT3lrpXiW'

using browserid credentials isn't resulting in browserid replication with sync_gateway

I'm using xhr to POST this to /_replicate, after successfully registering an assertion via a POST to http://lite.couchbase./_browserid_assertion and getting back my email in the response.

{"source":"http://animal.local:4984/sync_gateway","target":"mydb","auth":{"browserid":{"email":"[email protected]"}}}

In theory it should then use the assertion to create a session, on next replication. But on next replication I get the same 401 error, and my sync_gateway logs look like:

2013/03/07 15:49:22 GET /sync_gateway/_local/2086f4c9a107f081aa8569ac8af3238b514cc6c0
2013/03/07 15:49:22 Auth failed for username="", cookie=<nil>
2013/03/07 15:49:22     --> 401 Invalid login

I can think of a few places stuff could go wrong. My big guess is that somehow setRemoteDictionaryValue isn't getting called when the JSON is parsed by parseReplicatorProperties.

I'm confused about how to move forward b/c the relvant code is spanning the public and private APIs.

Another possibility is I biffed the JSON embedded above, I already caught one mistake there, maybe there's another.

CBLModel cannot be used (much) without setting a database

I've got a subclass of CBLModel and I'd like to init without a database and set state. However, an assert in setValue:ofProperty: fires indicating that the _document ivar needs to be non-nil. If CBLModels can be used without residing in a database it seems this assert should be removed. If CBLModels must have a database connection then it seems they shouldn't be init-able without one.

Failed to sync document with attachment - 400 Bad Request

anybody can help me here? i cannot sync the document with attachment on iOS device to the sync_gateway.

Here is the error:
CBLMultipartUploader[PUT http://ec2.amazonaws.com:4444/test/5352BC98-3856-4FB5-BCA0-5159F654BFC0?new_edits=false]: Got error Error Domain=CBLHTTP Code=400 "400 bad_request" UserInfo=0x8e2bbd0 {NSURL=http://ec2.amazonaws.com:4444/test/5352BC98-3856-4FB5-BCA0-5159F654BFC0?new_edits=false, NSLocalizedFailureReason=bad_request, NSLocalizedDescription=400 bad_request}

query endkey format for composite keys

I have a view that emits a composite key with three elements, each element is a JSON node with two properties e.g. three compound keys might be:

[{"prop1":"foo","prop2":"bar"},{"prop1":"yee","prop2":"haa"},{"prop1":"key","prop2":"jar"}]
[{"prop1":"foo","prop2":"bar"},{"prop1":"jane","prop2":"doe"},{"prop1":"key","prop2":"jar"}]
[{"prop1":"blue","prop2":"car"},{"prop1":"john","prop2":"dee"},{"prop1":"wee","prop2":"flea"}]

I want to write a query that will pull out all rows that have a first key of:

{"prop1":"foo","prop2":"bar"}

My start key is working:

Query.startKey = @[@{@"prop1":@"foo",@"prop2":@"bar"}];

But I can't figure out how to write an end key that will match 'any' key value in column two and three, this is what I have based on reading the wiki and couchbase server docs for compound keys.

spoQuery.endKey = @[@{@"prop1":@"foo",@"prop2":@"bar"},@{},@{}];

When I add this, no rows are returned.

Seems that despite the docs, CBL is using the string match approach for end key.

In the example above a working end key can be created by appending \ufff0 to the end of the complex key:

spoQuery.endKey = @[@{@"prop1":@"foo",@"prop2":@"bar\ufff0"}];

This works because of the way the SQLite query is built.

Is this working as designed, or should it be working as per the Wiki docs?

Replication of a document which has an attachment with a "revpos" > the document rev prevents the rest of the document from being replicated

We recently started using HTTP COPY to to clone some Cloudant server databases, which contained documents with attachments. After doing so, we can no longer replicate these documents via CBL. The problem appears to have been discovered a while ago:

https://issues.apache.org/jira/browse/COUCHDB-1518

In CBL, we get the dreaded "400 Invalid attachment". And the document itself fails to replicate entirely. The part about CBL reporting the problem seems legitimate. But shouldn't this just be a warning, allowing the document to be replicated anyway, sans the offending attachment(s)?

Jens > "You're probably right, since such documents exist in the wild. I think this was just sanity-checking code I added to detect if anything was wrong with the metadata."

Registered blocks aren't shared between threads

If you instantiate multiple CBLDatabases on a single database file, they mostly stay in sync because they operate on the same documents. But map/reduce/validation/filter functions registered with one aren't seen by the other.

This causes problems when you start a CBLListener, because it creates new CBLDatabase instances on its background thread. HTTP requests to the listener that involve views or filters or validation won't work correctly.

need a blocking call to _replicate

I can't currently come up with a reliable way to detect the need to authenticate. My current heuristic is to check _active_tasks periodically and look for a 401 status. However sometimes I get a status that looks successful, before the error status. Or other times the replication has status Offline until the connection is established a fraction of a second later.

If the POST to _replicate didn't return until the replication either succeeded or failed definitively (or timed out...) then I could more easily know when to prompt for credentials.

Support Cocoapods

It would be nice if the Couchbase development team could keep a canonical .podspec file updated in the main repository as new versions are released.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.