orisi / orisi

Oracle and client source code [START HERE]

#Orisi - Distributed Oracle System

Orisi is a distributed system of oracle nodes which validates smart contracts safely. Such contracts, unlike blockchain-only ones, can rely on external conditions.

Basic Bitcoin and Ethereum contracts cannot - by themselves - use external inputs, like stock prices or checking website urls for data. Orisi solves this dependency problem by creating a distributed network in which the majority of oracles have to agree to have a transaction validated. This distributed system makes it exponentially harder to bribe or otherwise influence the oracles, and is still able to validate a contract if one or more of the oracles fail.

Orisi Tutorial

Completing those steps should leave you with your own contract, and a thorough understanding of the Orisi distributed oracles system.

1. Read the White Paper - Introduction to Orisi
2. Create a timelock transaction - A thorough explanation of how to install the client and create a timelock transaction
3. Understand how the timelock works - a Khan-academy style video explaning how the process works
4. Create your own contract in 29 easy steps
5. Ask questions! - Questions, ideas? Create a new issue on Github

All the above steps should take you 10-20 hours.

What's where

Contribute to the code base! The main programs are split into the oracle and client folders, with a shared folder including other resources:

• src/oracle - Oracle node source

• src/client - Reference client source

• src/shared - Shared resources

---

Please keep in mind that both the client and oracle are in very early alpha versions and are unstable. We encourage you to use them and test them. Please post any issue to the Issue Tracker. If you have any fixes do not hesitate to pull request

orisi's Issues

Oracle should report all request problems via BitMessage

e.g. "required field Locktime missing" is now thrown onto a console

secrets.sh causing error in bitmessage config

When installing without the vagrant box, running secrets.sh and then starting up bitmessage causes an error, as bitmessage complains that the config file already exists. If secrets.sh is run after bitmessage, there is still an error as bitmessage complains that there are too many entries for the configuration file.

It may be best to have secrets.sh start up bitmessage once to initialize the configuration file, and then afterwards append the created api passwords to the file.

Planned protocol changes in BitMessage

https://bitmessage.org/wiki/Protocol_specification_v3

tl;dr; POW will be easier to compute with messages having short time to live & max message size will be 64kb now. A good opportunity to ask them to change something in the protocol in the discussion.

Can attacker outrun the contract user by monitoring blockchain

Let's say that I want to create a timelock.

1. I create a multisig address protected by all the oracles (let's assume that it's not protected by me - because other contracts like bounty aren't)
2. I wire in the funds
3. I get the transaction details, and send transaction request via bitmessage
4. Oracles create a timelock, and release the funds to a provided address when the time comes.

The problem is - if an attacker is monitoring a blockchain for all the multisig addresses, could he outrun me between the points 2 and 3? He would notice funds happening at a multisig address, and send his own transaction request to the oracles.

Two possible solutions to this:

1. have a point between "1" and "2", where send a "address-lock" request to the nodes, tying a multisig to my BitMessage address. This should be quite easy to implement.
2. Replace points two and three. So, I create a timelock before sending funds to the multisig address. This would require oracles to monitor blockchain, so it's much more expensive. On the other hand - if the transaction request is malformed, it wouldn't end up with locked funds as it's happening right now.

The point 2 we could implement for now by utilizing blockchain's API on the oracle's side. It's centralised, and it opens us up for a possibility of a ddos (creating millions of transactions and our oracles asking blockchain.info a million times for a confirmation), but it's very easy to implement...

any specific reason for using plural 'transacitons' in conditionedtransaction?

I'm changing the contract to use single transaction, but perhaps there was a reason as to why you wanted more than one transaction here?

Move wiki to this repository?

I'm not sure if two repositories were made for a reason, but would it be better to have the wiki and source in one repository?

Normalize naming.

guesspasswordhandler -> bounty_redeem_handler
conditioned_transaction_handler -> condition_create_handler

in general, we want the first part of the name refer to the contract name (BOUNTY, and CONDITION), the second to be a specific command name

Make sure that there's no option for an oracle to guess it's public IP from within docker

This includes not only checking settings outside of the box, but connecting to the internet not through TOR. Because if you can connect to the internet not through TOR, then you can call a service that gives you back your IP address...

signatures_number doesn't verify the signatures in bitcoinclient.py

Do I see correctly that the method we're using to verify the amount of signatures is this:

  current_signatures = len(asm_elements) - 2

Doesn't this code mean that an evil oracle (EO) could possibly block a system by publishing a transaction with more script parameters than necessary?

If there's no easy way to verify which sigs were used to sign the transaction, perhaps we could put out metadata along the Oracle's communication, containing information about who signed what?

Putting this into a freezer for now, as our topmost priority for now is better system design.

Using python floats with creating request in client

Function create_request from src/client/client.py uses both Decimal and float when handling values. I think it can lead only to troubles, but I don't see any easy fix as floating numbers are in general very hard to maintain

    outputs = {}
    for oracle in oracles:
      outputs[oracle['address']] = float(oracle['fee'])
      amount -= Decimal(oracle['fee'])

    if amount <= Decimal('0'):
      print "tx inputs value is lesser than fees"
      return

    outputs[receiver_address] = float(amount)

It lead to a problems like numbers 0.0018700000000000004444... in transactions. I've gotten rid of that by setting getcontext().prec=8 for Decimal, but I'm not sure if that is enough.

Self-contained Orisi Oracle?

I'm not sure if this is possible, but it would be very nice to have everything needed for an Orisi oracle to be in one folder and be able to run concurrently with any other processes, including other personal bitcoind and bitmessage instances. If it is possible, the Node could spawn it's own instances of Bitcoin and Bitmessage on unused ports separate from the defaults and be able to run on any computer.

The only issues I have are:

1. Is it possible to set custom config file locations for Bitcoind and Pybitmessage? That way the host machine personal clients could be left alone
2. Are there ports we can use we know for sure would not be in use?

Add "ping" functionality to the oracles

Each oracle should send a bit message to the network containing:

• protocol / source code version / checksum
• it's public key, and it's name
• fees that it requires

Was I right to remove an unnecessary optimisation when filtering tasks?

Check out this commit:

e1169f4

Correct me if I'm wrong, but the only situation when self.oracle.task_queue.get_similar(task) returns anything else than [task] is when more than one signing task is received within one second?

And even if more than one was received in one second, most would still be filtered away by if signs_for_transaction > signatures_for_this_tx:

Thanks to removal of this code we no longer have a situation where filter_tasks messes up with the queue of the task that was given in a parameter.

Separating TIMELOCK & ConditionedTransactionHandler

If I understand correctly, right now the transactions are being signed by ConditionedTransactionHandler. And all the other contracts, once the time of signing comes, create a ConditionedTransaction.

The problem with this approach is that ConditionedTransaction also contains a TIMELOCK contract built in. It makes the code quite convoluted.

What I want to do is create a separate TIMELOCK contract, which uses ConditionedTransaction as a signing mechanism. And rename ConditionedTransaction into TransactionSigner.

What do you think, @Gricha ? Would you like to tackle that this weekend?

Merge self-contained

I've set up a branch which installs PyBitmessage and Bitcoin inside the Orisi folder and then runs both instances on different, arbitrary RPC ports. This allows you to run the Orisi oracle (or client) alongside personal installed copies.

As of now I'm using the default http ports for Bitcoin and PyBitmessage as it didn't seem to cause a problem, but those can be changed as well.

Review commits from when you were away

I made some big changes to your code, in at least two cases I provided a detailed description on why the changes were made -- began them with "@Gricha". Go and find them.

Here's one of them:
05aff65

Oracle 187 keeps displaying "someone was faster" over and over again in logs.

Oracle .187. keeps displaying "someone was faster" over and over again in logs.

2014-07-02 09:05:20,277 [MainThread ] [DEBUG ] 0 new requests
2014-07-02 09:05:20,525 [MainThread ] [INFO ] someone was faster
2014-07-02 09:05:21,592 [MainThread ] [DEBUG ] 0 new requests
2014-07-02 09:05:21,841 [MainThread ] [INFO ] someone was faster
2014-07-02 09:05:22,908 [MainThread ] [DEBUG ] 0 new requests
2014-07-02 09:05:23,181 [MainThread ] [INFO ] someone was faster
2014-07-02 09:05:24,223 [MainThread ] [DEBUG ] 0 new requests
2014-07-02 09:05:24,509 [MainThread ] [INFO ] someone was faster
2014-07-02 09:05:25,585 [MainThread ] [DEBUG ] 0 new requests
2014-07-02 09:05:25,834 [MainThread ] [INFO ] someone was faster
2014-07-02 09:05:26,913 [MainThread ] [DEBUG ] 0 new requests
2014-07-02 09:05:27,162 [MainThread ] [INFO ] someone was faster
2014-07-02 09:05:28,226 [MainThread ] [DEBUG ] 0 new requests

Perhaps the message doesn't get marked as "read?"

Attacker can outrun create_bounty and ask for a different passphrase for the same multisig

I want to create a bounty. Right now I have to wire the money into the multisig address first, and then publish a bounty request.

In theory, the attacker might hear my bounty_create, and immediately publish his own, for the same multisig address, just with a different (known to him) passphrase.

There are two solutions to this:

1. Require that the bounty_create is signed by the client's signature in the multisig address.
   • pros: provides generic protection from attacker changing transaction parts
2. Require that bounty_create is a two-step process. First I announce the bounty for a given multisig, and then - once oracles confirm that the requested password is indeed the one I intended, I can wire in the funds, and deliver prevtxs to oracles.
   • pro: protects not only from this attack, but also from a situation when Oracles reject the transaction for any reason

We will probably want to implement both solutions.

review timelock_contract, transactionsigner & client/main.py

The refactor is complete! You might want to look at how timelock_contract now flows.

Let me know in this thread if you have questions

Use blockchain's API as an input for the client

e.g.

https://blockchain.info/rawtx/d0d41f7ec8435ed65ed078facbd92a9684021751f76735de2d8457dfa5a24050

Setting up without Vagrant?

Will there be a way to run this without installing the vagrant box? So far I have tried simply running the python program, but get errors regarding Passwords.

Installation procedure improvements

The install process is currently not very easy and has a few bugs and quirks. I recently created created a new box which uses a different directory structure, a git-maintained version of Orisi, and two scripts to easily setup and run the oracle node.

I think the best approach to installation may be to include only one script with Orisi in the root directory which installs Bitcoin and Bitmessage inside the Orisi folder and then does all the configuration. We can warn extensively that the computer running the node cannot already be running either process already and suggest that a simple vagrant box be set up, but the entirety of configuration and installation should be maintained in this repository, not in a large, pre-made vagrant box.

removing json_out check?

Can you check this out: 89a42a7

Was there a reason for checking json_out there? The only reason I see would be to allow for accepting the same transaction with the same outputs and timelocks to the same multisig multiple times. But I don't see where's a benefit to that?

(I know this is nitpicking, but I really want to polish the contract code, as people will be using it as a base for their own code)

code cleanup

Walk through functions & classes in BitcoinClient and other places, and see which of them are still being used. I'm quite sure we don't need most of the database classes any more.

The reason I'm pushing for refactoring & cleanup so much is that this is Open Source, with programmers coming & going, and every extra line of code makes it harder for them to jump & understand the architecture.

Client installation tutorial

We could really use installation instructions for the client, explaining how to install one on a computer, and how to perform a simple timelocked transaction.

errors reported by oracles should contain msig address/txid and be json-formatted

so we know which transaction is being referenced.

additionally there could be error "kind" reported. "retry" meaning that the request was simply rejected, while "blocked" meaning that funds are blocked on the account (e.g. because the bounty for that multisig was already requested and locked in.)

fyi, moved oracle_protocol.py into handlers.py

If I add a new contract, I want just one file to put in all the information about that.

(close the ticket to acknowledge)

Can you explain to me what decodescript does?

I have a hard time grasping the concept of decodescript, and what redeemScript is, and how is it different from p2sh.

I made this commit a16db96 but later on rolled it back.

Was the commit valid? Can you explain to me what decodescript, redeemscript, and p2sh are? I can't find this information anywhere on the web.

oracles should broadcast hashes of all the handler codes & their main code

Two reasons:

• a very slight security improvement
• easy way to track if oracles updated code to the new version

Moving vagrant scripts into /scripts folder on github

I've been fooling around with it, and it seems like it could be possible. The smaller download sizes and easier updates may make it a more viable choice.

Change install.sh, so all bitcoind installs and config is in ~, not in $dir

I've had issues* with bitcoind config files and/or binaries being outside of ~ folder. Can you please change the install.sh and runoracle scripts in such a way that bitcoind would land in ~, not in ~/orisi ?

• - when running vagrant, we want to make the ~/orisi/ folder shared with the outside world. but this causes conflicts with bitcoind. Moving bitcoind to a folder which is not shared with the outside world fixes the problem.

remove class RESPONSE

And other situations where you split messages away from the code. We're not doing internationalization, we don't need that.

Marking this as "yesterday" because it's easy to do, and will make the code much easier to read.

Bitmessage web passthrough

Perhaps it would be a nice idea to setup a bitmessage passthrough on a website (oracles.li?), available either with a user interface, or a REST API.

If we had this, a client could send messages to the network without having to have a bitmessage installed. This would benefit developers wanting to build on top of our network, as it would be much easier for them to create web clients for existing contracts.

(credit to Alex for this idea)

What happens if three BOUNTY guesses are being sent at the same time?

Imagine that we have just three oracles, and three people send a request each at the same time.

Oracle A gets Request 1
Oracle B gets Request 2
Oracle C gets Request 3

Would that cause the lockup of funds in the current architecture?

Fortunately we have a timelock, so this wouldn't be a big of an issue, but it's important to look out for such problems in future.

Clean up installation and scripts

@Gricha, I saw that you added a few scripts to easily start up the client and oracle, but it might be best to standardize how each is installed.

I rewrote the install and run scripts to use a PYTHON_EXEC environment variable that one can easily change depending on what python version their OS is running. Install.sh installs bitcoin and bitmessage and moves some configuration files, and run.sh starts up each instance and runs the oracle. Is there a reason for the start_oracle.sh and client.sh scripts? or can we just have a client.sh script that starts up bitcoin and bitmessage?

the client tests are now removed

The commit below pains me a bit, but may hurt you more :)

I know you spent a lot of time on them, but right now they are totally unfunctional due to the modified client's architecture.

They have a plenty of interesting funcitonality (generating mockup oracles, mockup multisig addresses etc), so you'll probably want to revisit this revision, and restore the files from it.
8bcea36

Once the tests reflect the new architecture, we can readd them, ok?

Compressing the transactions to make them standard?

"As for the current rules, make sure you're compressing your keys and including a sufficient fee for the transaction size. IIUC you should currently be able to do 15 on the current network without becoming non-standard. I doubt that will be increased any time soon because signature operations are expensive and potentially cause DoS risk."

https://bitcointalk.org/index.php?topic=645589.msg7413090#msg7413090

Apply for a Satoshi Award

Once we're ready.

http://satoshiawards.com

Oracles' BM addresses are now meaningful and persistent // add signing messages with Oracle's public keys.

( cc @83tb re: persistent data to stay on docker )

I found a simple hack of the BOUNTY contract.

Let's say that I publish a bounty_create request.
Now, the oracles should reply with new_bounty, and provide their RSA public keys, so people can send them the encrypted passwords.

But what happens if there's an attacker listening to a conversation that replies with his own "new_bounty" acknowledgement? How does the client know that it was one of the oracles replying, and not the attacker?

I solved this by publishing Oracle's BM addresses in the contract charter (charter="default node list"). But the proper solution would probably be to sign all the oracle's outgoing communication with it's bitcoin address. That way we'd be free to migrate away from BM easily, and also there would be less data about oracles to store in the charter.

@Gricha, what do you think? I think we discussed a similar solution some time ago, right?

RunOracle to quit if Bitcoind or Bitmessage failed to launch

Can you fix runoracle.sh to exit when Bitcoind or Bitmessage fail to launch?

Otherwise, there's a situation when the script tries to launch a node even when bitcoind failed to be executed. This makes it harder to debug for people, because the bitcoind error is displayed, but then quickly oracle's debug messages show on screen and hide it.

stop sending the whole message in reply to bounty_create

Right now, the oracle replies by sending the full content of original message back to the sender.

• is that really necessary?
• we could possibly force oracles to send huge messages this way, and lock them into computing the proof of work
• we could probably force messages to broadcast almost any message

What we could do instead is:

• send message_id back to the sender (so he knows in relation to what this is)
• possibly also send a hash of the original message to the sender (so he can be sure that it's a reply to his message, and not to a spoofed msgid - but this should be solved some other way really)

Why we need to use Vagrant or Docker

Right now the install is geared towards Vagrant, but Kuba is working on moving it to Docker soon. Why am I pushing for a virtual environment so hard?

Let's imagine that someone unknown to us wants to create a new Orisi contract. There are two ways to do that:

• let him submit a pull request for that contract. we'll then have to audit the code, do a pull, and push the new code onto all the oracles
• create a special scripting language for writing contracts, and a dedicated virtual machine for that -- that's the way Ethereum does it btw.
• let him just fork the Orisi node, do whatever kind of contract he wants, and contact us, so we can set up a node on all the servers, besides the original node.

The first solution would be extremely limiting. All the pull requests would have to be audited, and if someone managed to break the code, he could destroy all the other contracts. This is no way to do it.

The second solution requires a ton of engineering. Even if we just forked a code from another project, it would take a lot of time. And would still be insecure - just look at all the bugs that are constantly being found in JVM.

This leaves us with the final solution - that is, letting people fork Orisi, and setting up a virtual machine for every fork. This solution protects all the parties - the contract devs don't have to worry that someone will break the Oracle the contract is running on (because they're running their own codebase), and we don't have to audit every single contract.

To say all this in other words: imagine that there's Bart and Joe who want to develop Orisi contracts. What's the best way to allow them to develop such contracts, and run those contracts on the participating servers?

Or in other words still: we cannot do it without a virtual machine, because we plan to run a hundred Orisi instances on each Orisi server. And we want each of those instances to be running in a sandbox.

Let's discuss. It's important that we all get this straight, so any questions will be welcome.

Are bitmessages using non-default ports a security threat?

If all our Bitmessage instances run on a non-default port. Even if we changed it back to default, Vagrant (and probably Docker too) would forward it onto a different one on the main machine.

Isn't that a simple way to find out where are our nodes located? By getting a list of Bitmessage nodes, and limiting them to the ones with nondefault ip addresses?

Any ideas on solving this?

Reflect upon Handlers.py

Look at the code below. This is from handlers.py. Reflect for a moment on how you could make that code cleaner :)

handlers = {
     'conditioned_tranaction': ConditionedTransactionHandler,
     'bounty_create': PasswordTransactionRequestHandler,
     'guess_password': GuessPasswordHandler,
}

class OPERATION:
   TRANSACTION = 'conditioned_transaction'
   BOUNTY_CREATE = 'bounty_create'
   GUESS_PASSWORD = 'guess_password'

VALID_OPERATIONS = {
     'conditioned_transaction': OPERATION.TRANSACTION,
     'bounty_create': OPERATION.BOUNTY_CREATE,
     'guess_password': OPERATION.GUESS_PASSWORD
}

OPERATION_REQUIRED_FIELDS = {
     OPERATION.TRANSACTION: ['transaction', 'locktime', 'pubkey_json', 'req_sigs'],
     OPERATION.BOUNTY_CREATE: ['prevtx', 'locktime', 'message_id', 'sum_amount', 'miners_fee', 'oracle_fees', 'pubkey_json', 'req_sigs', 'password_hash', 'return_address'],
     OPERATION.GUESS_PASSWORD: ['pwtxid', 'passwords']
}

Re-organize scripts and decide upon install method

So now that Vagrant caused a hiccup with the install process and two different scripts had to be created, the root of the repository is very crowded and it is harder to understand and follow through with the install procedure.

I think that at this point, it might be best to suggest installation in a person's home OS, without vagrant or docker. Now that the processes run on separate RPC ports and can run concurrently with personal instances of Bitcoin or Bitmessage, and the fact that all configuration files are stored inside of the Orisi folder to allow easy control and removal, I don't see any downsides to using this way.

Also, I think it would be best to keep all OS-specific commands, e.g. apt-get which is Ubuntu and Debian only, outside of the scripts so that they can be used on Macs and other Linux distros. Extra dependency installs can be detailed in the README instructions.

The best structure I believe would be as such for now;

• install.sh - A script for someone who is running *nix and wants the oracle or client on their regular OS (no vagrant). This should be like the original install script I had and install all the config and binary files inside the Orisi folder and run on separate RPC ports.
• vagrantinstall.sh - A script identical to the one above but with config and bin files in the home directory to avoid the issues that were being brought up. This can stay until someone with more expertise can figure out what the problem was and fix the regular install script to work with both regular and vagrant installs.
• oracle.sh - Simple script that checks for running instances of Bitcoin and Bitmessage, starts them if they are not present, and runs the oracle.
• client.sh - Simple script which you can pass commands too, does the same checks as the above script

For the Docker install, I'm not sure if we should include the files, or just link to the separate repo. We need to decide whether to include the Docker install completely (for example inside a docker folder) or to simply link to the external repository in the README.

Lastly, I think that to lower the size of the install, the script should delete everything but bitcoind, as it appears to be that is the only binary that is needed

These are just some ideas I've been thinking about, feel free to change them or add ideas

vagrant box not synced with repo, installation problems

Idea of vagrant installation is for people to install it fast and painless. It's not like that yet.

vagrant@precise32:~$ ./secrets.sh
./secrets.sh: line 3: /vagrant/src/oracle/settings_local.py: No such file or directory
./secrets.sh: line 7: /vagrant/src/oracle/settings_local.py: No such file or directory
./secrets.sh: line 13: /vagrant/src/oracle/settings_local.py: No such file or directory
done.

/vagrant/scripts/runbitcoind.sh
-bash: /vagrant/scripts/runbitcoind.sh: No such file or directory

Problems:

A) Default location of the scripts in the vagrant box(http://oracles.li/orisi.box) is /home/vagrant not /vagrant/scripts
(vagrant box add orisi http://oracles.li/orisi.box)
I will check with the "mega" image also, but seems really messy.

B) Default vagrant box needs a script that will fetch the current installation from the github repo, so there is no config AT ALL in the box itself

C) Where is the originating source of the vagrant box? Who has this?

bounty_create fixes & testing

bounty_create is still to be fixed up after the refactor

Both timelock and bounty can be set up on the same multisig address

Split into repository for client and oracle

Eventually it might be better to have the client and oracle in two separate repositories, especially since they cannot be run on the same computer really.

After polishing client we need to add tests.