hoodiehq / hoodie

:dog: The Offline First JavaScript Backend

hoodie

A generic backend with a client API for Offline First applications

Hoodie lets you build apps without thinking about the backend and makes sure that they work great independent of connectivity.

This is Hoodie’s main repository. It starts a server and serves the client API. Read more about How the Hoodie server works.

A good place to start is our Tracker App. You can play around with Hoodie’s APIs in the browser console and see how it works all together in its simple HTML and JavaScript code.

If you have any questions come and say hi in our chat.

Setup

This setup is working for all operating system, testing on Windows 8, Windows 8.1, Windows 10, Mac and Linux.

Hoodie is a Node.js package. You need Node Version 4 or higher and npm Version 2 or higher, check your installed version with node -v and npm -v.

First, create a folder and a package.json file

mkdir my-app
cd my-app
npm init -y

Next, install hoodie and save it as dependency

npm install --save hoodie

Now start up your Hoodie app

npm start

You can find a more thorough description in our Getting Started Guide.

Usage

hoodie can be used standalone or as a hapi plugin. The options are slightly different. For the standalone usage, see Hoodie’s configuration guide. For the hapi plugin usage, see Hoodie’s hapi plugin usage guide.

Testing

Local setup

git clone https://github.com/hoodiehq/hoodie.git
cd hoodie
npm install

The hoodie test suite is run with npm test. You can read more about testing Hoodie.

You can start hoodie itself by using npm start. It will serve the contents of the public folder.

Backers

Become a backer and show your Hoodie support!

Official Sponsors

Show your support for Hoodie and help us sustain our inclusive community. We will publicly appreciate your support and happy to get your word out as well, as long as it aligns with our Code of Conduct.

License

Apache 2.0

hoodie's Issues

hoodie.my.* -> hoodie.*

Here's a list of current modules we have

Core:

// core
hoodie.my.store
hoodie.my.config
hoodie.my.account
hoodie.my.remote

//extensions
hoodie.user
hoodie.share
hoodie.global
hoodie.email

Why not kick the .my and go with the following?

// core
hoodie.store
hoodie.config
hoodie.account
hoodie.remote

//extensions
hoodie.user
hoodie.share
hoodie.global
hoodie.email

If I remember it correctly, we introduced the .my prefix to make it clear that all the modules are me-centred. But no I think: they all are me centered, so why not simply kick it? Backend is where we have to handle multiple users, but in the frontend, there is always only one user who's logged in, isn't it?

how to persist state not-yet-saved before signout?

STRs:

• sign in to demo app
• quickly type in a bunch of new tasks (10 or so)
• quickly sign out.
• sign in again

the set of tasks that will have been saved can be a subset of the set of tasks entered. Somehow, the app needs to "persist to disk" before signing out. Filing here rather than in demo app as I expect there's a bigger architectural discussion to have about how to make that happen.

remove dependency on jQuery

hoodie.js currently depends on jQuery for pragmatic reasons. It mainly uses jQuery's ajax and promise implementations.

prevent push after signout

when I change an object and signout within the 2 second timeout of store:idle, local changes get pushed to remote, which leads to a 401 error.

A push should not happen if the connection to remote has been disconnected

clarify `authenticated` & `unauthenticated` events

follow up for #44

use when.js for promises

We currently depend on jQuery for promises. Time to move on.
https://github.com/cujojs/when

safari browser user state refresh issues

user logins/logouts in the default app do not refresh the browser contents using the Safari browser under 10.6.8.

one has to do a manual reload to see that state has changed.

Migrate To PouchDB

hoodie.js is currently using LocalStorage for local persistence. It's slow, synchronous, and several people advised me not to use it for anything beyond a cookie functionality.

On the other side, there is http://pouchdb.com/. It works with indexdDB, but there is also a webSQL adapter. What kept me from using PouchDB is the lack of support for IE < 10.

But, I just found out that the most simple fallback is to use PouchDB, is to use it directly with the CouchDB endpoint. So when a user saves a document, the update gets directly sent to the Couch, instead of storing all documents locally in the browser and then asynchronously replicate with the Couch in the backend.

Also, when thinking about browser support for hoodie: IE < 9 does not support CORS, which is a requirement for the current default setup of a hoodie app.

It's definitely something we should consider.

add timestamps to _users docs, so we have a signUp date

remove $ prefixes from standard properties like type and timestamps

$type should become type, $createdAt simply createdAt. I thought it would be a good idea to mark these properties as "system propreties". But that's tech-centric. And we are user-centric. And when working on hoodie apps, I find the $ prefixes for these properties rather confusing, as I want to use them in my apps.

Error when doing "hoodie start" on fresh project

I followed the installation instructions and started a project with "hoodie new". I cd'd into the directory and tried "hoodie start". Here's the terminal output.

I confirmed local-tld is running re: http://twitter.com/hoodiehq/status/321415285519298560

I'm really excited about the project, so I hope you can help me out. 👍

findAll inside signIn callback doesn't return any results

hiya, this code doesn't return any documents:

hoodie.account.on('signin', function() {
  hoodie.store.findAll('email').done(function (objects) { console.log(objects) })
})
hoodie.account.signIn('test', 'test')

but if I wait a second and then run hoodie.store.findAll('email').done(function (objects) { console.log(objects) }) again then all the documents show up

Make a DevTools extension

http://addyosmani.com/blog/devtools-extensions-for-webapp-developers/

hoodie.store doesn't trigger idle events for dirty objects on startup

To fix it, hoodie.store should bootstrap all data initially

Mechanism to prevent abuse of email feature

Making sending emails from the client is great, but it also makes it easier for spammers to abuse the functionality.

I don't know how this is prevented in normal business applications. Here some ideas of mine:

• add the notion of "type" for emails. when setting up the backend email server, it can be setup how to ensure this type of email is not abused. E.g. there could be some upper max limit of emails per week/month or some captcha service should be used. Also, the type of email might have a predefined template and when sending the emails the client can only fill in the missing pieces.
• the hoodie-client.js API has a way to easliy setup a captcha in a div element
• as the developer should know what the email type expects, he should know if it's necessary to solve a captcha first before sending an email or not

The idea of email-types look handy to me to have some generic mechanism to define the emails send during the welcome, confirm and password-lost process.

camelize all methods and attributes

created_at --> createdAt
changed_docs --> changedDocs
etc

We want to follow the ECMA Script conventions

destroyed usernames cause db conflict upon re-use attempt

when a user account is destroyed (in the default app), re-use of the username causes a db conflict

add spec for Hoodie.Store#destroyAll

hoodie.store.clear should not wipe out entire localStorage, but only its own objects

rename events to be in present form

"created" --> "create"
"destoryed" --> "destroy"

etc .

We just discussed it with @janl, @espy and @colegillespie

other opinions?

Docs don't match up to the actual state of things

• Some events are missing (#43)
• Remove store.save
• Explain shares and the current limitations (no easy discovery)

Anything else?

rewrite hoodie.js in JavaScript

tl;dr

• replace Cakefile with grunt for building / testing / etc
• rewrite hoodie.js with all core modules to JavaScript, test against existing specs
• rewrite specs to JavaScript

This is a great opportunity to set the cornerstones of the future hoodie.js, your chance to become core contributor. The only requirement is to remain the frontend API.

Leave a comment if you're interested.

---

hoodie.js is currently written in CoffeeScript. The reason is, that I'm very comfortable myself to write CoffeeScript that documents itself pretty well. And at some points, when the code would become too complex, I'd rather keep it simple, readable, but therefore less performant.

I think that there can live multiple implementations of hoodie.js among each other, even for the same backend. There are other libraries that have that, like underscore.js or mustache.js.

But the main implementation of hoodie.js should be in pure JavaScript, I'd still keep the CoffeeScript implementation as learning code for the people that like it.

This is a huge chunk of work. But it's also a great opportunity to gather several developers that want to reimplement hoodie.js in pure JavaScript, and compine their experience to make it super solid. And the coolest thing: the specs are already there!

Comment if you're interested to help building it.

make `hoodie.extend` accept URLs for scripts & return a promise

it would be cool to have hoodie.extend to accept URLs of JavaScript files that can be loaded asynchronously, like

hoodie.extend('http://debug.hood.ie/script.js').then( function() {
  hoodie.debug.showLocalStore()
})

CouchDB connection refused

When starting the hoodie app just created using ‘hoodie new’, all HTTP requests to api.appname.dev return:

An error has occurred: {"code":"ECONNREFUSED","errno":"ECONNREFUSED","syscall":"connect"}

After entering an admin password passoword, hoodie crashes because the connection was refused with an invalid JSON response (starting with A).

Below is hoodie’s log.

$ hoodie start

> [email protected] start /Users/sander/Code/appname
> node node_modules/hoodie-app/lib/hoodie-app.js

Start local couch on port: 6004
CouchDB Started
hoodie server started on port '6003'
[api req] GET /
[api req] GET /
[api req] GET /
Please set an admin password: test[api req] GET /
[api req] GET /

undefined
setup done

open http://appname.dev in your browser

starting: 'worker-users'
initializing appname worker with: 
{ server: 'http://couch.appname.dev:80',
  admin: { user: 'admin', pass: 'test' },
  persistent_since_storage: false,
  name: 'users' }
[users] [Setup] reading global config from modules/module/appconfig …
All workers started.
Your App is ready now.
[static req] GET /
[users] [Setup] reading user config from modules/module/users …

[users] SyntaxError: Unexpected token A 0 [ 'SyntaxError: Unexpected token A',
  '    at Object.parse (native)',
  '    at Request.db_response [as _callback] (/Users/sander/Code/appname/node_modules/hoodie-worker-users/node_modules/hoodie-worker/node_modules/cradle/node_modules/follow/lib/feed.js:121:17)',
  '    at Request.request.self.callback (/Users/sander/Code/appname/node_modules/hoodie-worker-users/node_modules/hoodie-worker/node_modules/cradle/node_modules/follow/node_modules/request/main.js:104:22)',

The couch.stderr file contains a couple of lines like this:
execvp(): No such file or directory

I built CouchDB using homebrew, but installed Node.js using the Mac installer. Could that be the reason it fails (e.g. CouchDB not being able to find node)?

merge Hoodie.Remote & Hoodie.RemoteStore

from working with hoodie on some apps, I find it rather confusing to have a Remote instance which comes with methods to interact with the remote database, and then some more which are namespaced by store.

e.g. hoodie.remote.pull() vs. hoodie.remote.store.findAll()

I think we should merge these two modules and get rid of the .store namespaced methods

create separate issues for these

they are all related to the Hoodie.Account module

• authenticate should not return false if username is not set. There are
  circumstances when the username is not set, but the CouchDB session is still
  valid.
• make sure that same requests do not get sent multiple times, e.g.
  GET /_session should be only sent once. Instead of sending another request,
  the promise of the pending one should be returned. If that doesn't make sense,
  e.g. password reset, than we have to make sure that a running request gets
  aborted.
• spec window.setTimeout @authenticate in constructor
• what if @username is different from what GET /_session returns? Can there be a case like that?
• before sign in / sign up: sign out if hoodie.my.config.get('account.username') is a different email address

Add some docs for app.store.update if data changed in mean time

Looking over the code, I'm not 100% sure what's happening if app.store.update( type, id, {nr: 1} ) is executed but the same type and id was changed in the mean time on the server.

Is the object on the backend always overwritten with the latest object send by using the app.store.update api? Does the backend do some merging on the way? Is there a way to detect there is a conflict (e.g. does app.store.update fail?) and what's the best way to resolve it again?

(I think you gave me some of these answers this afternoon, but I'm not 100% sure anymore and some documentation would be helpful in general I guess.)

Exception after create admin password

I'm not sure whether this is the correct sequence when starting hoodie app. I got the issue below, it asked to create a new "admin" password, then follow by an exception:

$ hoodie start

[email protected] start /Users/tslsf/project/hoodie/mynewapptest
node node_modules/hoodie-app/lib/hoodie-app.js

Start local couch on port: 6004
CouchDB Started
hoodie server started on port '6003'
Please set an admin password: admin
undefined
setup done

open http://mynewapptest.dev in your browser

starting: 'worker-users'
initializing mynewapptest worker with:
{ server: 'http://couch.mynewapptest.dev:80',
admin: { user: 'admin', pass: 'admin' },
persistent_since_storage: false,
name: 'users' }
[users] [Setup] reading global config from modules/module/appconfig …
All workers started.
Your App is ready now.
CouchDB stop triggered by exit

TypeError: Cannot read property '_id' of undefined
at new Response (/Users/tslsf/project/hoodie/mynewapptest/node_modules/hoodie-worker-users/node_modules/hoodie-worker/node_modules/cradle/lib/cradle/response.js:13:14)
at Request._onResponse as _callback
at Request.self.callback (/Users/tslsf/project/hoodie/mynewapptest/node_modules/hoodie-worker-users/node_modules/hoodie-worker/node_modules/cradle/node_modules/request/index.js:142:22)
at Request.EventEmitter.emit (events.js:98:17)
at Request. (/Users/tslsf/project/hoodie/mynewapptest/node_modules/hoodie-worker-users/node_modules/hoodie-worker/node_modules/cradle/node_modules/request/index.js:856:14)
at Request.EventEmitter.emit (events.js:117:20)
at IncomingMessage. (/Users/tslsf/project/hoodie/mynewapptest/node_modules/hoodie-worker-users/node_modules/hoodienpm ERR! [email protected] start: node node_modules/hoodie-app/lib/hoodie-app.js

make hoodie test for existing connection

• make a method checkConnection on hoodie core module
• run checkConnection on startup
• checkConnection should simply do a GET <base_url>/, that should return {"couchdb":"Welcome","version":"1.2.1"} when connection works
• run checkConnection once per minute. If it fails, trigger offline event and check every 3 seconds.
• once it works again, trigger online event, and check every 60 seconds
• hoodie.checkConnection can also be run from other modules, for example when a request failed, like hoodie.remote.push()
• hoodie.remote should subscribe to online event and do hoodie.remote.push() when it gets triggered.

make hoodie.account.signUp more fault-tolerant

When hoodie.account.signUp or hoodie.account.anonymousSignUp because the account does not get resolved and the user reloads the page, hoodie should automatically try to finish the signUp.

on anonymousSignUp, "user/hash" doc gets created, instead of "user_anonymous/hash"

remote.on() doesn't return the changed object

remote.on() returns the object id as a string instead of the entire object.

Example:

hoodie.remote.on('changed', function(type, id, changedObject){
console.log("Woo: ",changedObject)
});

// -> Woo: ag0qdhr

implement public user stores

as described in the public store wish list API

implement hoodie.account.confirm(token)

At the moment, every new user that signs up gets automatically confirmed. To enable confirmation emails, we need a method to confirm an account with a token that gets send via email, e.g.

to activate your account, please click the following link:
http://www.myapp.com/#/activate/hash123

In my App, I'd read out "hash123" and then run hoodie.account.confirm( token )

Question now is: how could the CouchDB Magic work?

list of all events

things like .on('authenticated') don't show up on http://hood.ie/ or in the hoodie.js readme right now

rename hoodie.store.changedDocs() to hoodie.store.changedObjects()

... 'cause docs is to couch-y and we call it object everywhere else

simplify hoodie.share module

little brother of https://github.com/hoodiehq/worker-shares/issues/10

add spec for hoodie.share.open

also, make sure that hoodie.share.open passes correct options.prefix to hoodie.open

store.findAll().share : reject when no objects found

fix hoodie.store.find('task', '123').shareAt('shareId')

Currently, the following code will only add 'shareId' to the objects $shares has:

fix hoodie.store.find('task', '123').shareAt('shareId')

So it works fine if 'shareId' is a share that I created my self. But it won't work if it belongs to a share that I've not created but want to contribute to.

To fix this, the shareAt method needs to create a new share if it doesn't exist yet. Problems to think of:

1. share might or might not belong to me
2. I might or might not have access
3. I might or might not have write access
4. The share might not exist at all

My idea: make shareAt always create a new share if it's not in my local store yet. Set $createdBy to 'share/shareId'. Make the share worker handle the validation:

1. if share already exists and belongs to me, replace the new $share object with the existing one, create a new revision to trigger sync
2. If I don't have access to the share, set the $error property to the appropriate error. the local hoodie.share needs to listen to errors and act accordingly
3. If I don't have write access to the share, ... we must set some kind of flag on the `$shares.shareId' of all objects that are shared at 'shareId'. But there is no such thing yet.
4. If the share does not exist at all, behave as in 2., and then remove the $share locally, after triggering the error event.

password confirmation screen not used/ignored

reproduce: in the default app - decline to fill in the 2nd password field, account will be created anyway

undefined errors in account.signIn if XHR error

if api.appname isn't reachable you get undefined error message in the signIn callback, probably other callbacks as well

HTTP Request failed when tapping into Hoodie

MacBook-Air-2:~ rrichrs$ brew tap hoodiehq/homebrew-hoodie
Cloning into '/usr/local/Library/Taps/hoodiehq-hoodie'...
error: The requested URL returned error: 403 while accessing https://github.com/hoodiehq/homebrew-hoodie/info/refs
fatal: HTTP request failed
Ryans-MacBook-Air-2:~ rrichrs$

'authenticated' event fires multiple times

when I have a hoodie.account.on('authenticated' listener it calls the callback 3 times when I do hoodie.account.authenticate

hoodie.account.sign_up should sign_out first

Calling sign_up() while a user is already logged in returns a 404 and throws an error.

Expected behavior would be to sign out the current user and then seamlessly sign up the new one.

hoodie.store.findAll() returns objects that are marked as _deleted

lastActiveAt timestamps for users

we want to have a lastActiveAt timestamp for users.

Current idea would be to update my _users doc each minute with a new lastActiveAt timestamp.

That would produce tons of revisions in the _users docs, not sure if that's feasible

move uuid() method to hoodie core module

npm/browserify?

can I do this?

var hoodie = require('hoodie')

my dev workflow is here: https://gist.github.com/maxogden/5147486

i'm basically trying to figure out how to split my hoodie app up into many node modules and use beefy as the dev server, rather than the built in hoodie start

clarify .save vs .add

is the only difference between store.add and store.save that you can specify an id with save?