Official Wallet Backend & API for Cardano-SL
Home Page: https://input-output-hk.github.io/cardano-wallet/
License: MIT License
Cardano-Wallet
See Wiki
We want users to be able to retreive new/unused addresses for accounts they own in their wallet. With the (ed25519)BIP-44 scheme, this can be done from the account public keys (as in non-hardened child key derivation) and through address pool api. Address pool contains gap of discovered unusused addresses and we can use this pool to inform the user about the unused addresses. Address pool might contain some used addresses as well so having optional (or default) flag to filter only unused might be nice to have.
Extend the API to support retreival of new/unused addresses defining a new endpoint for external wallets. The api will return list of addresses from the address pool. There will be optional (or default?) flag to filter unused only from the address pool. Implement this ticket in two interations:
must support retrieval of new/unused addresses for E.O.S. Walletsmust support retrieval of unused only addresses for E.O.S. Wallets| Number | Base |
|---|---|
| # | develop |
| Criteria | Coverage |
|---|---|
| ? | - |
With #33, we should have the necessary tools to perform account discovery while
applying blocks. Answering the question isOurs happens during pre-filtering
where we skim through incoming blocks, looking for inputs that are related to
our wallet(s).
Using the address pool module, we should be able to abstract away the account
discovery to use the relevant method depending on whether we have a F.O. wallet
or an E.O. one.
| Number | Base |
|---|---|
| #139 | develop |
| #179 | develop |
| #180 | develop |
| #181 | develop |
| #183 | develop |
| #192 | develop |
| #204 | develop |
This was a big one but has finally landed. The major and most important change is actually fairly simple but allows to abstract over the pre-filtering (resp. recovery) process by having the implementation working with a generic "IsOurs" interface.
which goes together with:
So, by having prefilterBlock works with any isOurs function that satisfies the interface, we makes it possible to support multiple scheme here and are free to answer the question the way we'd like (i.e. using a root private key like for F.O. wallets or a pool of account public key like for E.O. wallets).
An important thing to notice from the isOurs interface above is that it's seemingly stateful:
isOurs :: Address -> s -> (Maybe HdAddress, s)So, given a state s, we return a corresponding address and the new state. This plays nicely with our existing address pool module:
s is here the AddressPool address :) . Then, it's a matter of persisting and merging the state between queries. The statefulness is rather useless for F.O. wallets since the underlying state doesn't change (and is actually discarded by the BListener and the Restore modules). Yet, it's essential for E.O. wallets as they discover new addresses.
Note that, we do not yet provide an instance of IsOurs because we need to resolve #32 first and have our data-model sorted out for E.O. wallets. Yet, it'll be fairly easy to write once we have that. I'll create a corresponding ticket for both this and the restoration procedure to tight all this together once #32 and a couple of others have landed.
In terms of testing, since this was mostly a refactor of our exacting source code, we've made sure that existing tests worked in a similar fashion as before. Locally, I've also added some noise during integration tests to make sure prefiltering happened correctly (as suggested in #184 and explained in cardano-foundation/cardano-wallet#204 (comment))
In a second stage, we will want to sync with mainnet, restore an existing wallet and see what's going on here. I'd rather keep that for #40 or, once we have fully integrated E.O. to make it easier to setup.
This issue was much bigger than initially estimated. Our main mistake here was not to anticipate the amount of work needed in the prefiltering code to make it ready for BIP-44. So, instead of just implementing a new feature, we had to re-implement the base feature first.
This is symptomatic in more part of the code base actually, and there are still few modules that need some love. Therefore, this kind of redesigns / heavy refactorings should become an ongoing process to prevent code from rotting in the long run.
Here, we are looking for a module which can keep track of address pools and answer the question:
isOurs: Address -> Maybe (WalletId, AccountId)
This can be achieve in various way but a few pointers:
We can have an opaque AddressPool types which is implemented behind the scene as some specific
data-structure (like a Map); managing this structure is what the module does.
How to generate new items should be provided to the module (ideally, the module doesn't know about addresses).
We should have a way to initialize the module from a list of items to track.
FIXME: To be completed with design decision (@uroboros)
cardano-wallet should provide a new module to host Address Discovery utilities as explained in BIP-44.| Number | Base |
|---|---|
| 118 | develop |
| 129 | develop |
| Criteria | Coverage |
|---|---|
| Creation & validation of address gap | See Spec/AddressPoolGap |
| Creation, lookups and extension of address pool | See Spec/AddressPool |
$ stack test cardano-wallet:unit --test-arguments "--match Pool"
AddressPool
lookupAddressPool
+++ OK, passed 100 tests:
62% hit outside pool
38% hit within pool
AddressPoolGap
mkAddressPoolGap
+++ OK, passed 100 tests:
69% gap outsides boundaries
31% gap within boundaries
Part of the work done for defining external wallets was done without clear requirements on the underlying derivation scheme. Now that we have a brighter idea of what we need, we can adjust existing model types to welcome incoming changes.
The ExternalWallet type has to be extended to welcome the sequential derivation scheme which includes:
| Number | Base |
|---|---|
| #57 | develop |
| #78 | develop |
| #85 | develop |
| #113 | develop |
| #114 | develop |
Now we are able to create EOS-wallets from a list of accounts' public keys only. We cannot check if these account's public keys correspond to all accounts of this EOS-wallet (we even cannot check if they belong to the same EOS-wallet), so we don't make any assumptions about it and just work with these public keys with no relationship.
Since we don't have a root key for EOS-wallet, we use UUID to identify it.
Address pool gap (for new address derivation) can be specified on API-level, otherwise default value (based on BIP-44 specification) will be used.
This is finalizing the work started in #31 but including private key derivations and operations this time.
Extend the module defined in #31 to also support hardened derivation.
purpose' and coin_type' internal derivationschange internal derivationchange internal derivationcardano-cli to generate golden tests https://github.com/input-output-hk/cardano-cli, https://cardanorust.iohkdev.io/cli-wallet/cardano-cli instructions https://github.com/input-output-hk/cardano-wallet/wiki/Testing-Yoroi-and-Cardano-walletcardano-crypto lib, have a look at cardano-cliWord31
Word31covercardano-cryptoquickchecks arbitrarySizedBoundedIntegral| Number | Base |
|---|---|
| cardano-foundation/cardano-wallet#162 | develop |
| cardano-foundation/cardano-wallet#163 | develop |
| cardano-foundation/cardano-wallet#169 | develop |
| cardano-foundation/cardano-wallet#170 | develop |
| Criteria | Coverage |
|---|---|
| 1. | Has been covred with property tests (see bellow) |
| 2. | Has been covred with property tests (see bellow) |
| 3. | Has been covred with property tests. It is not explicitely tested but implicitely through property equals to deriving address private key and extracting public part from it: N(CKDpriv((kpar, cpar), i)) === CKDpub(N(kpar, cpar), i) |
| - | Additionally this PR fixes address public key derivation from account public key (previous version wasn't bip44 compatible - it was missing internal change chain derivation). This PR adds missing change chain derivation and adds property tests (see bellow) |
$ stack test --ta '-m "Ed25519Bip44"' cardano-wallet:unit
Ed25519Bip44
Deriving address public key
fails if address index is hardened
+++ OK, passed 100 tests.
succeeds if address index is non-hardened
+++ OK, passed 100 tests.
equals to deriving address private key and extracting public part from it: N(CKDpriv((kpar, cpar), i)) === CKDpub(N(kpar, cpar), i)
+++ OK, passed 100 tests.
Deriving address private key
fails if address index is hardened
+++ OK, passed 100 tests.
succeeds if address index is non-hardened
+++ OK, passed 100 tests.
fails if passwords differ
+++ OK, passed 100 tests.
succeeds if passwords are equal
+++ OK, passed 100 tests.
Deriving account private key
fails if account index is non-hardened
+++ OK, passed 100 tests.
succeeds if account index is hardened
+++ OK, passed 100 tests.
Finished in 186.8338 seconds
9 examples, 0 failures
There was a cardano-crypto bug discovered (IntersectMBO/cardano-crypto#52) during this issue which is great - but it took some time to debug and fix it upstream. Property tests are not very strong (in my opinion) and some are very weak (there are opened issues that should tackle that). Also, most of property tests in this issue should be moved to cardano-crypto (where they could have prevented the unforseen bug). The actual surface for mistake is very small within this issue (as most of heavy lifting is done by cardano-crypto) so implementing additional tests for this issue should not be considered as high priority (in my opinion).
There are multiple issues opened that can improve upon this issue (see development plan for links)
Historically, we created the cluster package inside the cardano-sl repository to have an easy way to run cluster of nodes. However, at this time (and still today), edge nodes and wallet backend server are tightly coupled. Starting one also starts the other which is rather unpractical as it creates some circular dependencies between cluster and cardano-wallet.
We need to remove the dependency from cluster to cardano-wallet and have the wallet started separately. This is quite tricky because, at the moment, the wallet still needs to boot an actual node.
cluster musn't have any dependency to cardano-sl-wallet-newcardano-sl-cluster-demo to preserve compatibility.| Number | Base |
|---|---|
| cardano-sl#3929 | develop |
| cardano-sl#3935 | develop |
The cluster package now only relies on cardano-sl code and edge nodes are now just edge nodes; without any wallet.
Because we haven't completed decoupling yet we however still need to start wallet with their underlying edge node. To achieve that, a new executable tool as been created as part of the cluster code to generate a cluster environment (which we can use to start a wallet).
This also removes the circular dependency between the wallet and cluster package which enables us to complete #117 and run integration tests using this cluster!
BIP-0044 Multi-Account Hierarchy for Deterministic Wallets is a Bitcoin standards defining a structure and algorithm to build a hieararchy tree of keys from a single root private key.
It it built upon BIP-0032 and is a direct application of BIP-0043. It defines a common representation of addresses as a multi-level tree of derivations:
m / purpose' / coin_type' / account' / change / address_index
Paths with a tild ' refer to BIP-0032 hardened derivation path (meaning that the private key and passphrase are required to derive children). This leads to a couple of interesting properties:
This allows for external keystores and off-loading of key derivation to some external source such that a wallet could be tracking a set of accounts without the need for knowing private keys. This approach is discussed more in details below.
NOTE:
One other important aspect is more of a security-concern. The introduction of
such new address scheme makes it possible to change the underlying derivation
function to a new better one with stronger cryptographic properties. This is
rather ad-hoc to the structural considerations above, but is still a major
motivation.
Instead of mixing up the sequential derivation with new scheme in the existing deriveLvl2KeyPair and related. We'll instead create and test in isolation a BIP-44 modules with a few helpers to achieve key derivation.
Note that initially, there's no need to support derivation related to private keys, this is out-of-scope for E.O.S wallets.
cardano-wallet should provide a new module to host BIP-44 related utils.| Number | Base |
|---|---|
| 133 | develop |
Address indexes are now created in sequence from the last known address
Users should be warned if they try to create more addresses than the address gap.
We may also simply forbid it by default.
We only need the parent account's public key; the wallet root private key isn't
needed anymore.
FIXME: To be completed.
| Number | Base |
|---|---|
develop |
Final step towards completing the (ed25519)BIP-44 derivation scheme implementation. We need to unify / finalize the restoration procedure to support the following matrix:
| Address Derivation Style | Private Keys Ownership | Created / Restored From | |
|---|---|---|---|
| E.O.S. Wallets | Sequential | No (External) | Accounts Public Keys |
| F.O.S. Wallets | Sequential | Yes (Keystore) | 15-word+ Mnemonic |
| F.O.R. Wallets | Random | Yes (Keystore) | 12-word Mnemonic |
Unify API to support restoration of both sequential and random wallets through a list of mnemonic. We'll make explicit the type of wallet being restored by adding an extra (optional) API parameter. The number of supported mnemonic words will be extended from 12 to: 12, 15, 18, 21 and 24 with an optional mnemonic passphrase.
Note that, the wallet type parameter should be optional, and defaulted to random (to keep backward-compatibility). Also, the API should reject any restoration of random wallets with more than 12 words and/or with a mnemonic generation passphrase (as those aren't supported features for random wallets).
| Number | Base |
|---|---|
| #? | develop |
| Criteria | Coverage |
|---|---|
| ? | - |
The current API for creating unsigned transactions works with F.O.R wallets only. This is slightly useless for exchanges as the whole point of doing off-wallet signing is to avoid communicating private keys to the wallet. Yet, since F.O. wallets do own their private keys (by definition), it defies completely the purpose. In the end, there are still use-cases where we might want to create unsigned transactions from a F.O. wallet, but the focus is primarily on E.O wallets.
Ideally, we would like to work for both E.O and F.O. wallets since no private keys are needed for this operation (that's the whole goal).
| Number | Base |
|---|---|
develop |
This can't be achieved without the corresponding address private key. When the private key is owned by the wallet, there's not much change in the process apart from choosing the right ed25519 implementation. This imply that this information should be somewhat available in the wallet or corresponding account and passed (at the moment) onto the deriveLvl2KeyPair function. Now, it's a matter of picking the right derivation function for the underlying wallet / addresses.
Modify existing code to support both derivation scheme in order to derive the correct address private key from a wallet root key.
deriveLvl2KeyPair https://github.com/input-output-hk/cardano-sl/blob/e9daaa6c98433fefa0d9cd4dd5578f80353eefa1/core/src/Pos/Core/Common/Address.hs#L387)mkSigner method to handle both derivation scheme cases (with conditional)
mkSigner dirrectly and/or via creating a new transaction. They should work with Addresses of both derivation schemesverifyKnownInputs - specificly checkWitness/checkSig https://github.com/input-output-hk/cardano-sl/blob/e9daaa6c98433fefa0d9cd4dd5578f80353eefa1/chain/src/Pos/Chain/Txp/Toil/Utxo/Functions.hs#L247 to work with both derivation schemes
Test.Spec.Ed25519Bip44
WalletActiveLayer.pay
Testing mkSigner view new transaction might be by:
newTransaction (similar as we do it within unit tests https://github.com/input-output-hk/cardano-wallet/blob/07def6f6789d0797105a1a473de70617ca3d1a18/test/unit/Test/Spec/NewPayment.hs#L199 ) . This will use mkSigner to create TxAux which contains witnessesTxAux) with verifyTxUtxo which will call verifyKnownInputs internallynewTransaction >>= verifyTxUtxo should return Right with Addresses of both derivation schemesMy intuition is if mkSigner returned Right SafeSigner, then witness validation will pass as signing happens regardless of which derivation scheme we use (as signing algorithm doesn't depend on key derivation scheme). That said, having the above test would be endgoal - but until we get there we might test that mkSigner is expected to return valid SafeSigner, where "valid" in this context is defined as:
mkSigner should return SafeSigner that contains XPrv created by old derivation scheme (not sure can this be verified by cardano-crypto lib)mkSigner should return SafeSigner that contains XPrv created by new derivation scheme (not sure can this be verified by cardano-crypto lib)mkSigner requires db snapshot populated with keys of new derivation scheme| Number | Base |
|---|---|
| 209 | develop |
| 223 | develop |
| 249 | develop |
We want the Wallet to be able to receive blocks as they get validated by the underlying node.
The Wallet is interested in the blocks for several reasons:
It maintains its own UTxO via a set of specified ledger rules (see Formal specification for a Cardano Wallet)
It tracks a bunch of Protocol Parameters that are specified in the genesis block and may
change via updates. For the Wallet, the relevant parameters are:
The parameters are currently not extracted from the block but queried directly via some IO using
the fact that Wallet runs as a node.
Knowing how to transition from an initial state via applying a block, it would be possible for the
Wallet to keep track of these parameters as it applies blocks (see A Simplified Formal Specification of a UTxO Ledger).
There's a major question about rollbacks and how to "un-apply" rules.
Ideally, we want the ability for the Wallet to ask for two things:
An initial state (derived from the genesis block by the node) where state can be
many things (e.g. protocol parameters, system start, stake pools distribution, utxo)
The current state (for fast syncing)
Then, a consumer can apply ledger rules they're interested in as they receive blocks,
assuming node has validated pre-conditions for applying those rules. Having the initial
state and the transitions is all we need to keep state in sync.
Before taking any further action, we want to comprehend and document the relevant parts of the Node-To-Node protocol that is used to receive blocks. A few areas to start looking at are located in the chain and networking packages from cardano-sl as well as the Kernel/BListener module from cardano-wallet.
See:
We've moved the wallet into its own repository as a first step towards decoupling. Now, the cardano-wallet uses cardano-sl as a peer dependency and rely on it for quite a few things. However, a few modules in cardano-sl also relies on code inside the wallet. This creates some circular dependencies between both repo.
One of those dependencies is actually the BIP39-ish implementation from the cardano-wallet used in a few scripts and executable currently located on cardano-sl.
We want to reverse this dependency and have the BIP39 implementation part of cardano-sl. Thought, this can exist on its own as a mnemonic package. Let's avoid bip-39 qualification after all since our implementation is only inspired from bip-39 but deviates on a few points.
Also, let's relocate a few executables and scripts inside this mnemonic package (for instance, the "generate mnemonic" utility in scripts/)
cardano-sl should have a new mnemonic package.mnemonic library implementation should be taken from src/Cardano.Wallet.Kernel.BIP39.hs.test/Cardano.Wallet.Kernel.BIP39Spec.hs should be moved inside this new mnemonic package.mnemonic should provide a replacement for scripts/generate-mnemonic.hs.Cardano.Wallet.Kernel.BIP39 across both cardano-sl and cardano-wallet should be replaced appropriately.scripts/generate-mnemonic.hs should be replaced appropriately.| Number | Base |
|---|---|
| input-output-hk/cardano-sl/#3854 | input-output-hk/cardano-sl/develop |
| #63 | develop |
scripts/generate-mnemonic.hs had no dependencies and could be moved to stack exec cardano-generate-mnemonic without trouble
Aside from that, the issue was straight-forward, and went according to plan.
We need to prepare testing of external wallets. It's not unrealistic to have it separated from the rest.
Let's not go to fancy here, but a few property tests with some basic BDD.
See #117
| Number | Base |
|---|---|
develop |
Being a node, the Wallet "shortcuts" the transaction submission by directly pushing transactions
to the node's queue. In order to fully decouple the wallet, we would need the wallet to use
the Node <-> Node protocol in order to submit transactions.
Before taking any further action, we want to comprehend and document the relevant parts of the Node-To-Node protocol that is used to submit transactions. A few areas to start looking at are located in the chain and networking packages from cardano-sl.
https://github.com/input-output-hk/cardano-wallet/wiki/Submitting-Transaction-Byron
The wallet component comes with diffusion layer component and we want to continue with this. Diffusion layer component takes care of submitting transaction using infra and network primitives and disseminate it to the peers. We understand it fully and even when ouroboros-network people introduce something new we expect they adapt the diffusion layer with their new developments an we are still happy.
Until here, we didn't need the remaining classic CRUD operations on E.O.S. wallets.
For completeness and because we want to make our users happy, let's implement (and test) them.
| Number | Base |
|---|---|
| #278 | develop |
| #304 | develop |
FIXME: To be completed with outcome from #15
FIXME: To be completed with outcome from #15
| Number | Base |
|---|---|
develop |
Our current unit tests are only testing F.O.R. accounts and we might need a bit of fiddling in the arbitrary generators to also test our new derivation scheme.
FIXME: To be completed.
| Number | Base |
|---|---|
develop |
It has become more and more difficult to work efficiently on cardano-sl. Many teams cross-submit work from there, uses different workflow, have their own needs in CI and prevent each other from progressing at a nice pace.
We've decided to decouple the development of the wallet from the development of cardano-sl. This starts with moving the versioning of the code in a separate repository where we are a bit more free to operate.
wallet-new from cardano-sl into its own repository cardano-wallet.stack.yaml file allowing developers to easily build the project locally..cabal file to accomodate the new structure.| Number | Base |
|---|---|
| #23 | develop |
We haven't been able to introduce git sub-modules in the same time in cardano-sl. Thus as the moment, maintaining the wallet-new is a bit hectic because we have to "backport" every change made from cardano-sl into cardano-wallet and vice-versa. This is even more true because we are in the middle of a release (2.0.0) and we keep discovering bugs in the new data-layer therefore fixing them in cardano-sl/develop, cardano-sl/release-2.0.0 and cardano-wallet.
It also took quite a while to setup the CI with reasonable time and checks. Now that it's working correctly, we can already see the benefits of working on a separated repo, away from the noise of caradno-sl. Hope it will truly pay off eventually.
Cluster has a dependency on the wallet's servant client. It will need to be fixed to point at the code's new location. At the moment, cluster (and a couple of other packages) lives on cardano-sl whereas, they aren't so-to-speak part of the settlement layer. This package may therefore be re-located in its own repository, or in a more appropriated one like https://github.com/input-output-hk/cardano-shell.
| Number | Base |
|---|---|
develop |
In cardano-sl, it has been sometimes difficult to work our way with the CI. It is complex, mostly built in nix and uses many services (e.g Hydra, AppVeyor, Buildkite). In addition, because it has to accommodate for many needs across cardano-sl, it is rather slow (since any change in the wallet code would for instance trigger builds and tests of all other parts of the code).
Because the system is complex, it also fails for rather unexpected reason and ops time is required in order to fix it. This is rather inefficient for developers are powerless when it comes to managing issues with the CI.
cardano-wallet will run its own CI for testing on Travis and have complete control and management over it. The idea is to move the responsibility of testing back to developers. Dev-Ops still have responsibility to build and ship the final software but they'll be able to use whatever tool they like to do so.
This way, both teams should be able to work in parallel without interfering on each others.
cardano-wallet sources, and run tests in CI.hlint in CI.weeder in CI.Much harder than first anticipated. Building cardano-sl isn't actually that easy, especially with limited resources. We could have made things easier perhaps by requiring a budget for Travis-CI in a first place but went all-the-way with the free version granted for open-source softwares.
In the end, in "usual" mode (i.e. when we aren't bumping the LTS or a cardano commit), the CI runs in around ~15 minutes and gives us a rather fast feedback compare to before.
Although setting up Travis was quite challenging (in particular, having concurrent jobs running), it now gives us some good foundation to add more jobs and checks to the CI without, hopefully, increasing too much (or at all) the CI times.
We had to pick up "osx" instances to cope with the installation of rocksdb on Ubuntu VM. However, Ubuntu VMs of Travis are provided by Google Cloud and are slightly faster, especially when it comes to starting jobs (Mac VMs sometimes hang for 10 minutes before actually starting the job). So, we might reconsider these bits of the CI and find a workaround to setup rocksdb on Ubuntu VMs.
In order to avoid breaking the API versioning contract, we are required to support the Node API in the V1 of cardano-wallet. The V2 of the wallet API will not need to support it.
Since both APIs are identical, it is really straight-forward to use one to implement the other. Though, we want to make sure that calls go over the wire as-if the two processes were decoupled.
cardano-wallet backward-compatible node monitoring API should solely rely on a Node's API and relay requests through HTTP(s) to the nodecardano-wallet| Number | Base |
|---|---|
| #160 | develop |
| #196 | develop |
| #210 | develop |
| Criteria | Coverage |
|---|---|
| 1. | Manual verification that node-related endpoints delegate to the Node API. Integration tests have been written to ensure that the wallet can do this successfully |
| 2. | The code for a node client was written, but in the cardano-sl repository. |
Final step in the implementation of E.O.S wallets. This should be fairly straightforward assuming the work done in Pre-filtering and with the address pool module.
We want to re-use existing restoration code but only abstract away the bits related to account discovery (isOurs).
| Number | Base |
|---|---|
develop |
Faucet has a dependency on the wallet's servant client. It will need to be fixed to point at the code's new location. At the moment, faucet (and a couple of other packages) lives on cardano-sl whereas, they aren't so-to-speak part of the settlement layer. This package may therefore be re-located in its own repository, or in a more appropriated one like https://github.com/input-output-hk/cardano-shell.
FIXME: To be discussed with ops.
FIXME: To be defined with ops.
| Number | Base |
|---|---|
develop |
At this stage, we should be able to restore external wallets using a sequential derivation scheme like those created from Yoroi. This makes room for cross-application testing!
| Number | Base |
|---|---|
| #? | develop |
This is a bit special for E.O. wallets because, it's not really about "creating" a new account. We just can't because we need the root private key for that (hardened child derivation). Note that, restoration is totally out-of-scope here. So it's really just about providing the API capabilities and making the corresponding acid-state changes. On the long-run, we want to kick an async restoration after adding new account public keys.
We'll provide endpoint to give users the ability to add new accounts to a wallet. At this stage, we do not require any account index even though this could be useful for faster restoration.
Restoring historical data of such account is left for later.
| Number | Base |
|---|---|
develop |
FIXME: Complete this decision once #10 & #11 have been resolved.
| Number | Base |
|---|---|
develop |
Discuss whether to store account public keys in acid-state or in a dedicated keystore. Evaluate pro and cons and go for the best.
NOTE: In case the keystore option is considered, have a look at: https://github.com/input-output-hk/cardano-sl/pull/3808/files
FIXME: To be completed with the outcome of the discussion.
| Number | Base |
|---|---|
develop |
We first started by experimenting with Travis and Coveralls; yet building cardano-wallet is quite a great challenge as this requires first to build cardano-sl. We want to have reasonable CI times, so there was quite a lot of fiddling with Travis.
We've faced a few challenges with Travis:
Using the free version, we are limited to 50 mins job time. This is unfortunately not enough to build the stackage snapshot from scratch. However, Travis allows for a build to be split across as many job as we want. So as a work-around, one can spread a long task across many jobs.
Apart from pure Haskell dependencies, cardano-sl also requires a few system libraries. In particular, couchDB is a pain in the *** to install on the linux VM on Travis.
Travis has some caching capabilities which work out-of-the-box. However, Travis select caches depending on a few parameters, including: the branch and the environment variables set up. So, we gotta be careful with that.
Caching .stack-work isn't really a good idea as files in there changes almost with each build. Hence, Although Travis makes a diff on each cached folder before attempting to re-upload them (it doesn't if there's no change), we have been wasting a lot of power to re-upload a cached .stack-work as a result of running tests. Almost 1/3 of the build time was about downloading and uploading the corresponding cache.
We've a better understanding of Travis & Coveralls capabilities, we are going for the following approach:
We've defined a custom snapshot for our stack.yaml as cardano-sl.yaml. This is roughly the stack.yaml file from cardano-sl and allows for better caching on both our local and CI environments.
The build stages can be divided as such:
cardano-sl (and wallet-new)cardano-sl extra dependencies)cardano-sl required by the wallet.developThe first three steps above take time and aren't ran on each PR. They're triggered by specific messages and changes in the code that are fairly rare. All the build artifacts from this steps should be cached and speed up execution of next stages.
Installing rocksdb with linux is a pain on Travis, instead, we can boot an OSX VM and use homebrew to install this dependencies.
| Number | Base |
|---|---|
| #53 | develop |
| #54 | develop |
| #56 | develop |
The wallet has gone through quite a lot of changes recently with the introduction of the new data-layer. This includes some reshuffling in the way transactions are submitted.
We want to identify and document (as a wiki page) the process of submitting transactions in a cardano-wallet. This could come with schemas, code excerpt and prose to give as many details as possible.
The wallet uses its internal diffusion layer as an intermediary to submit transaction to the diffusion layer component. As the wallet and diffusion layer components are stitched together, and plan to be that way in the future, we are happy with how it works
FIXME: Complete with outcome from #12
Once #12 has been designed, we can move forward implementing the interface using the Node-to-Node protocol. If possible, we may want to add a few unit and integration tests on top of it.
| Number | Base |
|---|---|
develop |
Since Yoroi already implements the (ed25519)BIP-44 derivation scheme, they are a bunch of testing we can get from free by using it.
| Number | Base |
|---|---|
develop |
See #7
We'll follow a similar pattern as what was done for the wallet in terms of server, cli and tls authentication.
--tlsca and --tlscert to pass TLS certificates to the server.--no-client-auth to disable client certificate verification.--no-tls to completely disable TLS, in which case, certs aren't required.| Number | Base |
|---|---|
| cardano-sl#3882 | develop |
Change the explorer dependencies to point at decoupled code. This will be pretty much the same as #5
See #7
We'll move most of the implementation logic related to the API onto the node and have the wallet re-use this (since the wallet uses cardano-sl as a library).
No need to hook the handlers into a process / server at this stage.
cardano-sl -> cardano-wallet introduced in #8.1| Number | Base |
|---|---|
| cardano-sl#3820 | cardano-sl/develop |
| cardano-sl#3851 | cardano-sl/develop |
Now it's time to "complexify" our classic fully owned wallets to support either scheme (random or sequential).
We will also extend our BIP39 support to enable passphrase on mnemonic words. This isn't and shouldn't be possible for random wallet for backward-compatibility reasons. However, wallet using the new scheme should have this feature.
Also, it's probably a good idea to put hard-limits on both the minimal number of mnemonic words (12 sounds reasonable) and the maximum number (the current BIP-39 specs stops at 24; sounds like a reasonable top-bound which might be extended later).
FIXME: To be completed.
TODO: we have discussed that upon wallet creation we will add a tag/field that will discriminate between old wallet scheme and new wallet scheme (bip44, address changed, derivation scheme changed). We will use this information to fix how mkSigner checks a wallet type:
| Number | Base |
|---|---|
develop |
The wallet currently offers some basic monitoring of the node via the /api/v1/node-info and
/api/v1/node-settings. Underneath, this piggy-backs on the Node State Adaptor
and the fact that the Wallet is ultimately a node. This includes:
syncProgress: Syncing progression, in percentage.blockchainHeight: If known, the current blockchain height, in number of blocks.localBlockchainHeight: Local blockchain height, in number of blocks.localTimeInformation: Information about the clock on this node.subscriptionStatus: Is the node connected to the network and to what peers?slotDuration: Duration of a slot.softwareInfo: Various pieces of information about the current software.projectVersion: Current project's version.gitRevision: Git revision of this deployment.Strictly considering the wallet, these pieces of information are quite out-of-scope.
They should instead be provided by a node itself and queried by clients directly from
the underlying node.
At the moment, the Wallet is used as a kind of proxy to relay these pieces of information,
but in order to properly decouple the Wallet from the node, we should look into moving this
API onto the node, with a possible redesign as we do it.
In order to avoid breaking the API versioning contract, we will be required to support the Node API in the V1 of cardano-wallet.
The V2 of the wallet API will not need to support it.
Additionally, the Wallet also may require the systemStart to be available in
such API in order to compute a few things from it.
cardano-wallet, on cardano-sl.| Number | Base |
|---|---|
| cardano-sl#3788 | cardano-sl/develop |
wallet-new and cardano-node packages into lib. This was a relatively straightforward "pull the string" modification, where I copied code and didn't stop until the compile errors went away.Now that we can process blocks of transactions with addresses using the sequential derivation scheme à la BIP-44, we can try creating transactions through Yoroi and see whether they get picked up by Cardano.
A wiki page as been produced as a result, giving details about testing this with the Rust cardano-cli (same code powering Yoroi wallets).
https://github.com/input-output-hk/cardano-wallet/wiki/Testing-Yoroi-and-Cardano-wallet
Actual testing will be done as part of #239 once the corresponding features lands on develop.
We've initially focused on externally owned wallets and separated the definition and treatment of E.O wallets from F.O. wallets. Now that F.O. wallets also support the new derivation scheme, we have to adapt block pre-filtering to apply the right behavior depending on the underlying wallet's type.
FIXME: To be completed.
| Number | Base |
|---|---|
develop |
The testing situation on the wallet has historically been quite bad. With all the recent changes, we intend to improve our tooling in order to assess the quality of the code.
Code coverage falls into this category and we're therefore looking for hooking up https://coveralls.io to our CI. It offers nice reporting capabilities and seems rather easy to setup.
Setting up Coveralls on top of Travis was rather simple in the end, provided we ran all our testing on it. It gives us some nice reporting we can now use as a tool to identify dead code and possible areas where testing is missing.
Note: we aren't yet running integration tests as part of the testing, so results are a bit biased and misses a lot of coverage in the API / Handlers part.
One "oddity" I've noticed: because we use property testing in a few places, coverage somewhat varies, not quite a lot but still, from a commit to another even if the code is so-to-speak, unchanged. Because of the random nature of property based testing, we end up testing slightly different parts of the code on each execution. This isn't a problem per se, but let's keep that in mind and take coverage with a grain of salt.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.