comit-network / comit-rs Goto Github PK

As an exchange service, I want to request the current rate of trading BTC to ETH from the treasury service.

DoD:

• Able to generate request
• Parse response
• Mock Treasury service

Dependencies:

• API definition of treasury service
• Timeout of offers (delta/time wise)

As the trading service, I want to be notified about the deployment of the Ethereum HTLC with the contract address.

trading-service: implement secure call to be informed of the eth redeem contract address.
The trading-service will blindly trust this address as it will be provided by a blockchain watcher dedicated to the trading service.

• rename exchange folder to exchange-service
• ensure underscore is used in all package names

As a trading client, I can take the signed tx and forward to the trading service which will broadcast it.

DoD:

• take signed tx in stdin
• forward to trading service

We should only accept offers for trading BTC for ETH at the moment.

For this we need:

• a check in exchange service that rejects other symbols
• a proper error messages (there depends on #29)

As a trading service I want to forward a decline offer request to the exchange service.
As an exchange service I want to process decline offer requests and delete them locally.

DoD:

• Trading service generate message to exchange service to delete offer
• define delete offer on Exchange service API
• exchange service process delete request and delete the information (e.g trade identifier) for storage

re-order imports (or check why it's not working as reorder_import should be true by default)
see https://github.com/rust-lang-nursery/rustfmt/blob/effba718b4a46811d7baba185129b2eb34b3f126/Configurations.md

As a trading service, when the client accepts the offer I:

• Broadcast the transaction.
• Send the raw script + txid to the exchange service, so that the exchange service can validate the P2SH address.

As a trading service, I can receive an accept offer. This includes a signed transaction.

• Reconstruct the script for the P2SH address.
• Print everything to a log.

As a trading client, I want to generate a trade request to the trading service.
Once the trade offer is received, I want to parse it and ask the trader whether they want to accept or decline the offer.

DoD:

• Generate request trade as per trading service API
• Parse response (trade offer) from trading service
• prompt user

Create API documentation (Thomas mentioned OpenAPI) for each service. Need to be close to the code to avoid deprecation (in code? File at root?)

As the exchange service, after receiving the secret for the Bitcoin HTLC by an external service (being poked) I create a redeem transaction and send it to the blockchain.

Implementation hints:

In order to allow the exchange service to be decoupled from the blockchain, the exchange service itself does not literally watch it. Instead, it expects to be "poked" and thereby notified by an external service about this event.

This event needs to contain the transaction ID that funded the address, and the secret to redeem. At this point, we trust the external party to give us the correct data. As this is an "internal" endpoint, it needs to be secured later on.

As a Trading client, I want to be able accept an offer. I do that by

• creating a new transaction
• funding it
• signing it

and sending it to the trading-service.

Implementation hints:

For now, we only communicate with bitcoind via RPC. Needs unlocked wallet. CLI needs config for how to connect to the node (ENV variables). Ask user securely for password for unlocking the wallet.

As the trading service, when the client initiates a trade, I

DOD:

• generate a secret and store it (secret generation code is already in trading-service)
• generate an HTLC (BTC) and a P2WSH address

For the MVP we need a bunch of RPC methods.

• createrawtransaction [{"txid":"id","vout":n},...] {"address":amount,"data":"hex",...} ( locktime ) ( replaceable )
  • to create a new raw transaction, e.g. needed for almost all trades
• validateaddress "address"
  • to verify an address and retrieve its public key
• signrawtransaction "hexstring" ( [{"txid":"id","vout":n,"scriptPubKey":"hex","redeemScript":"hex"},...] ["privatekey1",...] sighashtype )
  • to provide input for p2sh and sign the raw transaction
• sendrawtransaction "hexstring" ( allowhighfees )
  • to submit the transaction to the blockchain
• getblockcount
  • to check which block we are at
• getrawtransaction "txid"
  • to retrieve a transaction, e.g., the transaction from the trader which the exchange was waiting for
• listunspent ( minconf maxconf ["addresses",...] [include_unsafe] [query_options])
  • to retrieve a list of unspent UTXOs which can be used as input for a new tx
• fundrawtransaction "hexstring" ( options )
  • to provide input for a raw transaction

Branch: https://github.com/tenx-tech/swap/tree/feature/btc-rpc-client
RPC-Command documentation: https://bitcoin.org/en/developer-reference#rpcs

Bitcoin Core offers a ZeroMQ endpoint (https://github.com/bitcoin/bitcoin/blob/master/doc/zmq.md) that can be used to subscribe to the following events:

• new block (hash)
• new transaction (hash)
• new block (raw)
• new transaction (raw)

Here is a working example that simply prints out every message to stdout:

https://github.com/tenx-tech/swap/blob/feature/zeromq-listener-bitcoin-core/bitcoin-zeromq/src/main.rs

I already put together some thoughts about how we could integrate this into the exchange and the trader application:

1. Currently, this uses a blocking loop in order to receive the messages.
2. If we integrate it into one of the applications on a source-code level, we have to use some kind of async mechanism. (Async in Rust is in a big overhaul atm - async/await keywords are coming)

My suggestions would be to externalize this thing into a dedicated program that simply translates these messages to HTTP calls. As we already have an HTTP API in the trader and the exchange, we can easily deal with these messages as well.

Further tasks:

Figure out the allocation of functionality: Where do we want to the filtering of interesting transactions? We only care about transactions of current trades.

Several possibilities:

1. Watcher forwards all transactions: Exchange/Trader have to filter for interesting ones.
2. Spawn one watcher per interesting transaction (parameterized through CLI parameter): Watcher filters for interesting transactions: Exchange/Trader only gets notified about interesting ones.
3. Watcher and Trader/Exchange communicate bi-directionally: Trader/Exchange post tasks to the watcher.

Open questions:

1. What if a watcher dies or is killed? How are they restarted?
2. As transaction hashes are unique, can they kill themselves after firing a notification? (Only relevant for option 2 above)
3. If the watcher keeps state (option 3), how do we deal with the async stuff?

Define the CLI interface:

Things that need to be configurable:

1. The ZeroMQ endpoint
2. The URL the messages should be forwarded to. (One for each type? One URL for all notifications?)

For option 2 and 3 we might consider embedding the trade-id into the URL, for example. A watcher could then be spawn like this:

./bitcoin-watcher --transactionhash=796d7a2dbb1213b65dc2f7170575755efdfae8340b2183e971ed5a89113bbedf --notificationurl=http://example.org/exchange/trades/7e641b0f-2f74-4a3b-a612-1aad89047203/transactions

A nice library for defining the parameters: https://github.com/TeXitoi/structopt

To accept the trade the user execute a "trade accept" command on the cli

He has to provide two addresses:

• Success address
• Refund address

For now, success is always an ETH address and refund is always an BTC address.

DOD:

• Implement command
• Read addresses into memory

Use http://docs-rs.com/http-api-problem/0.5.1/http_api_problem/ to generate good error messages.

If we activate the feature with_rocket, we can use HttpApiProblem as the Err type in Result responses from our handlers.

Child of #408.

As the trading service, when the trading client asks me for a redeem transaction, I will query my internal state to check if I have already received the contract address.

If there are enough confirmations (> 4) and there is still enough time to redeem the funds (> 1h), I will create a redeem transaction and send it back to the client.

In the case that there are not enough confirmations (or contract has not yet been created), we error to the user to wait.

In the case that there is not enough time to redeem securely, we tell the user.

Implement the new route-structure for accepting offers:

POST /ETH-BTC/buy-offers
{
    "amount": 1
}

This translates to the following:

Buy 1 ETH for x BTC.

The route should implemented in a 'hardcoded' manner. Means that the symbol is not(!) a path variable.

Todo:

• Refactor exchange service similarly to trading service to allow testing
• Update on the exchange service
• Update on the trading service

As a trader, I want to be able to decline the offer, because I am cheap guy and don't want to spend so much money for trading.

DoD:

• Take user input to decline offer
• send request to trading service to decline offer and delete offer
• Process decline offer on trading service and delete offer

Find out what needs be changed to go to testnet in the code (hard coded addresses, etc).
Setup a testnet node (bitcoin and ethereum) and get the money.
Find out, start doing and document it!

Do not worry about making it dynamic at this stage.

Steps:

• Run bitcoind testnest
  It is running on 35.189.58.109:18443 - Using the bitcoin docker image (/home/bitcoin/.bitcoin is persistent so we will not loose the data)
• Run eth(?) testnet
• Setup private keys for testnet
• Do full test with testnet

Definition of Done:

• successful swap on testnet
• documentation of what to be done in mainnet

After accepting a trade and inputting their addresses, the user needs to send the addresses to the trading-service.

DOD:

• Make http request to trading-service
• Accept request on trading-service (print it out)

As the trading client, when the trading service confirms the trade, I will:

• request the user to send the specified amount to the given address
• display a bitcoin-payment link
• display a summary of the trade information

DOD:

• print it out to the user

As the trading service, I'll forward an incoming request from the trading client to the exchange service within the same request.

As a result, I expect a trading offer from the exchange-service that contains an identifier along with the actual offer.

Upon receiving the offer from the exchange, I:

• generate secret
• add the hash of the secret to the response to the client

DoD:

• Message defintion
• Resource (URL) definition
• process request from trading-cli
• Generate request to exchange
• Parse and store response from exchange
• generate and hash secret
• send response to trading-cli

Impl-Hints:

Probably gonna be a POST request.

Update software to ensure that all steps support arbitrary amounts (float) and rates in safe manner.

Definition of Done:

• Swap can run with a rate of 0.075112 returned by the Fake treasury service (6 digits precision)

All amounts in our APIs should be represented as an object of the actual value and the associated currency. In the same step, we should default to serializing wei/satoshi. Regarding the JSON serialization, we should probably put the numbers in a string to avoid any kind of conversion/precision errors.

DoD:

• All amounts in APIs use the new format
• Happypath tests are working
• Serialization/Deserialization happens in one place (custom serializer and deserializer for BitcoinQuantity and EthereumQuantity)

Document the timings
ie, 12 hours & 24 hours.

ETH:

• exchange service decides timestamp 12 hours from now.
• exchange service tells trading service
• used in contract

BTC:

• 24 hours relative

Currently, most of the fields in the request/response structs are private because that is the default.

Where necessary during #5, I added the pub keyword so that I can move along. This is not a long term solution.

We need to figure out a consistent approach for data that needs to be accessed from the outside.

• from currency
• to currency
• amount (from currency)

DoD:
print out parameters

Scratches:

enum TradingSymbol {
    ETH_BTC,
    ...
}

DoD:

• Serializer
• Deserializer

asx --buy BTC --sell ETH --sell-amount 42 # ETH:BTC (sell 42 eth, buy ?? btc)
asx --buy ETH --sell BTC --sell-amount 42 # ETH:BTC (sell 42 btc, buy ?? eth)

notes

• The cli doesn't try to construct the symbol
• The trading service may deny it or pass it on to the exchange service which may in turn deny it

Add tests for all existing code

As the exchange service, after sending the HTLC-data to the trading service, I monitor the BTC blockchain for incoming deposits to the P2WSH address.

In order to be safe, I only watch for deposits to an address that I recomputed myself from the data the trading service sent me.

Upon a successful (proper amount & correct address) deposit, I will create an ETH-HTLC from the data I created earlier and deploy it to the chain.

Implementation hints:

In order to allow the exchange service to be decoupled from the blockchain, the exchange service itself does not literally watch it. Instead, it expects to be "poked" and thereby notified by an external service about this event.

This event needs to contain the transaction ID that funded the address. At this point, we trust the external party to give us the correct tx id. As this is an "internal" endpoint, it needs to be secured later on.

As the exchange service, if I receive a trade offer request from the trading service, I will:

• Forward that request to the treasury service
• Generate an identifier for the trade
• Store the trade somewhere
• Send the offer back to the trading client

DoD:

• Exchange service API definition
• Store trade in-memory
• Actually send requests and responses
• Response needs to contain the target address

As the trading service, when the exchange service responds with the HTLC (ETH) information, I will

• store this information locally
• include this data in the response to the trading client

What should be in the HTLC info:

• Exchange refund address
• ETH HTLC time

see: #53

There should be a shell script that automates the setup of the project:

• Install correct Rust nightly version
• Pin rust version for this directory locally
• Install rustfmt commit hook

Regarding the changes done to implement regtest in this 2 open source projects:

• Move the forks to coblox github
• cleanup the changes for rust-bech32-bitcoin
• create pull request on rust-bitcoin repo
• Wait for them make rust-bitcoin compatible with latest rust-bech32-bitcoin
• cleanup the changes for rust-bitcoin
• create pull request on rust-bitcoin repo

For Ethereum and Bitcoin, Swap should use appropriate fees. These can be hardcoded for now because for the mainnet swap, the only thing that counts is that they are ok.

• Ethereum: #106
• Bitcoin: #116

Ensure all private keys are in an env variable

As the exchange service, when the trading service proposes a trade, I:

• store info from trading service (later to be used to generate HTLC)
• generate an ETH refund address for the exchange service
• Calculate the timeout depending on the BTC max timeout from trading-service AND exchange service's minimum time needed to redeem.

suggested:
ETH_HTLC_TIME = BTC_HTLC_TIME - EXCHANGE_BUFFER_TIME_NEEDED

As the exchange service, I respond with the following information to the trading service, so that he can reconstruct the HTLC:

• exchange refund address (used for monitoring contract deployments)
• ETH htlc time

Note: The client already knows the secret and the success address.

DOD:

• include the above in the HTTP response

Use an env variable/argument to set-up the rate that the fake-treasury returns.

DoD:

• Rate can be set when starting the fake-treasury

Currently, there seems to be a bug in the derivation of the address from the private key.

Somehow, it works fine using ganache but doesn't work with an address generated through meta mask.

The following private key should generate the followed address:

Private Key: dd0d193e94ad1deb5a45214645ac3793b4f1283d74f354c7849225a43e9cadc5
Address: 0x33DcA4Dfe91388307CF415537D659Fef2d13B71a

As the trading client, when the user wants to redeem his funds, I ask the trading service for the redeem transaction, so that the user can sign and broadcast it.

Note: The trading client does not sign by itself.

DOD:

• command "redeem" to check if the particular trade is redeemable (by trade id)
• if it is, then the trading-service will respond with a redeem transaction, which the user needs to sign and broadcast

Spike:

• Check if it is easy to see whether new lines (from PR diff) are tested or not
• Check if test coverage can be ran on the e2e tests.

Notes:
Use cargo make
Add badge to readme
See https://testing.googleblog.com/2014/07/measuring-coverage-at-google.html?m=1

We need the pubkey hash for creating the HTLC

As the trading service, I send the following data to the exchange service so it can compute the P2WSH address as well:

• hashed secret
• client refund address
• redeem timeout (max acceptable time to have BTC locked up -- block height. Hardcoded)

In addition, I include the ETH success address so that the exchange service can create the other HTLC.

DOD:

• send HTTP request with the above data

Things to note:

• We have to use the nightly version of rust. We should install a dedicated one though

• We need docker to run a bitcoin-core node for the integration tests

  • It would probably be best to create our own docker image for doing that.
  • Make the docker image open-source and push it to dockerhub :)

• We should check the formatting

• Properly setup caching so the builds are fast

• Enable slack notifications instead of emails

• Require green builds for pull requests

Ideally, we can achieve something similar as with the bitcoin-core zeromq integration where we are notified about every new block/transaction that makes it into the chain.

Defintion of done for this ticket:

• Links to documentation about subscribe mechanism
• Implementation concept (used protocols & allocation of functionality)