trustbloc / edge-core Goto Github PK

In addition to the non-persistent simple database from trustbloc/edge-store#27, we need a real CouchDB implementation.

This will be useful for edv.

we will use https://github.com/igor-pavlenko/httpsignatures-go instead

If the user sets ErrGet on this store then the store's Get() operation should immediately return with that error.

This will support the functionality described in trustbloc/edv#49

The CouchDB storage provider's NewProvider() constructor does not perform a smoke test to see if CouchDB is actually ready to be used.

Warning: using the kivik client's Ping() method is not enough because that just queries the _up endpoint, which doesn't guarantee the calls to create databases will succeed. PR hyperledger-archives/aries-framework-go#2180 took the approach of querying for the existence of the _users database as a proxy to determine whether the CouchDB database is ready for use.

part of #75

• Add documentation for 0.1.1
• Release 0.1.1

part of #75

We need to validate the proof on the zcap-ld tokens.

I believe there's an SQL injection attack possibility here that needs to be addressed...unless this is somehow handled by the underlying library already. I found an article here that looks helpful: https://www.calhoun.io/what-is-sql-injection-and-how-do-i-avoid-it-in-go/

^ It describes how to avoid injection attacks via escaping (so you can still have your parameterized table name). Not sure if they're using the same library as you, but I assume there should be an equivalent if your library is different

#37 (comment)

Many projects can benefit from having the docker utility used in BDD tests implemented in this project and then reused.

Here is one example of the utility I'm referring to.

Needed for trustbloc/edv#7

The CouchDB provider type keeps a local cache of CouchDBStore objects. If a client calls provider.OpenStore(), the method checks to see if the store already exists in the local cache so it doesn't have to do another network call, which should save time. When closing a store via the provider, this store will be removed from the local cache.

However, if the underlying database in CouchDB is removed by an external force, then that cache will be out of date and OpenStore will return a CouchDBStore object that will fail consistently. provider.CreateStore() would need to be called again to make that CouchDBStore object functional again, or alternatively a new Provider object could just be created.

Is this ok?

Port the logging utility from Aries over to Trustbloc: https://github.com/hyperledger/aries-framework-go/blob/master/pkg/common/log/logger.go

Two reasons:

1. For consistency with the mem store and couchdb store implementations.
2. EDV needs the ability to attempt opening a store without automatically creating one if it doesn't exist.

Users may see this error after operating the mysql store for a while.

A likely cause of this problem is the implementation's use of the MySQL USE statement: it only applies to the connection in which it is executed.

Golang's sql/DB manages a pool of connections. Executing USE in one of them does not affect the rest. Also, connections in pools may be recycled after MySQL closes the connection due to idleness.

The implementation needs to ensure USE is ran on each connection that will be used to write or read data.

Related: trustbloc/adapter#295

• letter case for content identifiers: httpwg/http-extensions#1308
• which parameters are tokens and which are quoted-strings?: httpwg/http-extensions#1307
• http header line folding: httpwg/http-extensions#1306
• datatypes of parameters: httpwg/http-extensions#1305
• expected behaviour when encountering unrecognized parameters: httpwg/http-extensions#1304
• use of parenthesis around some content identifiers: httpwg/http-extensions#1303

• Add documentation for 0.1.2
• Release 0.1.2

This issue was generated by todo based on a TODO comment in 9da270a when #1 was merged. cc @DRK3.

Requirement

It is desired to enable an authorization strategy using a form of authorization capabilities where the sender's authentication and capability invocation are transmitted in HTTP message headers using HTTP signatures.

The capability invoker crafts an HTTP message with special headers indicating their KID, the content being signed over, as well as the actual signature itself. The sender's signing key is presumed to have been communicated out of band to the verifier.

Initial signing algorithm supported will be ECDSA using curve P-256 and SHA-256.

Note: we should not use (algorithm) since it enables attack vectors in some cases (eg. "ecdsa-sha256" section 5.1.2).

We will use igor-pavlenko/httpsignatures-go to create and parse HTTP signatures as per the HTTPbis draft.

Breakdown

• parse headers (including new headers) #79
• canonicalization of header fields #88
• sign and add Signature header using ECDSA with curve P-256 and SHA-256
• reusable middleware to wrap existing handlers #101
• API to create *http.Requests with signatures
• godocs + examples (examples can be found in the tests)
• PR httpwg/http-extensions#1297 will pull in structured headers. Will need to refactor. Parsers:
  • https://github.com/dunglas/httpsfv (Go)
  • https://pypi.org/project/http-sfv/ (Python)
• parse zcap-ld #92
• validate ld proof #95
• create API #97
• support for did:key #98
• httpsignatures.Secrets implementation for did:key (for verification) #105
• integrate httpsignatures-go with KMS #106

Known spec issues #89 :

• Missing documentation on certain fields of the JSONLD security vocabulary:
  • invoker: w3c-ccg/security-vocab#24
  • invocationTarget: w3c-ccg/security-vocab#25
  • parentCapability: w3c-ccg/security-vocab#67
  • allowedAction: w3c-ccg/security-vocab#67
  • capabilityChain: w3c-ccg/security-vocab#28
  • delegator: w3c-ccg/security-vocab#26

References

• Signing HTTP Messages
• GitHub Issues
• digitalbazaar/http-signature-zcap-verify
• digitalbazaar/bedrock-edv-storage

Support both Shamir secret sharing for use when splitting a secret.

When a document is deleted, couchDB does a soft delete and marks it at 'deleted'. If we attempt to fetch the deleted document with its ID, an error "Not Found: deleted" is thrown.

Currently, we do not take this into account when we check for errors from the getRevID() call which does not allow deleted docs to be recreated.

part of #75

spec: https://w3c-ccg.github.io/zcap-ld/
some examples: https://github.com/decentralized-identity/secure-data-store/issues/113

Known spec issues:

• Missing documentation on certain fields:
  • invoker: w3c-ccg/security-vocab#24
  • invocationTarget: w3c-ccg/security-vocab#25
  • parentCapability: w3c-ccg/security-vocab#67
  • allowedAction: w3c-ccg/security-vocab#67
  • capabilityChain: w3c-ccg/security-vocab#28
  • delegator: w3c-ccg/security-vocab#26

see: https://github.com/trustbloc/edge-core/blob/master/go.mod#L10
(and associated imports)

should be github.com/cenkalti/backoff/v4

I've been forced to update golangci-lint while working on #75 (because http.Request.Header.Value does not exist in whatever go version is shipped with golangci-lint 1.21) and found these errors:

pkg/storage/mysql/mysqlstore.go:166:30: rows.Err must be checked (rowserrcheck)
        rows, err := newDBConn.Query(
                                    ^
pkg/storage/mysql/mysqlstore.go:318:31: rows.Err must be checked (rowserrcheck)
        resultRows, err := s.db.Query(findQuery)
                                     ^
pkg/storage/mysql/mysqlstore.go:388:30: rows.Err must be checked (rowserrcheck)
        rows, err := indexStmt.Query(s.tableName)
                                    ^
pkg/storage/mysql/mysqlstore.go:383:32: Rows/Stmt was not closed (sqlclosecheck)
        indexStmt, err := s.db.Prepare(getIndexStmt)

This task is a followup to fix these linting issues.

A custom logger has its own logging levels and mechanisms to initialize those levels, so a custom logger should not have to use the logging levels defined in edge-core.

Would be good to have a Delete() method on the store interface.

Use case: transient storage, where a state machine needs to delete state to ensure a particular instance of the machine proceed properly on towards the next state.

part of #75

Need to serialize CapabilityDelegation properly as JSON-LD documents before proof verification.

It would be nice to have an API that looks like this:

// mysql provider automatically selected 
provider, err := storage.Open("mysql://root:my-secret-pw@tcp(127.0.0.1:3306)/")
if err != nil {
    // handle error
}

This would reduce refactors users would have to do when selecting different storage providers.

Current obstacles:

• import cycle b/w storage package and implementation-specific packages
• need to pass in implementation-specific options. Perhaps encode these options as url parameters?

MySQL identifiers.

Need to use the backtick to protect against illegal characters that may be present in the user's store name.

This will be used in the EDV project so that it can avoid recreating indices unnecessarily.

The CouchDB store always uses Context.Background(). Change this so that the context is specified by the caller somehow.

This issue was created from #7 (comment)

CouchDB is on version 3 now. Update Kivik, tests, etc and anything else that might need updating to support CouchDB 3. One notable change is that they got rid of "Admin Party mode", so some tweaks will need to be made to the tests as currently there isn't a proper admin user set up.