jewel / clearskies Goto Github PK
View Code? Open in Web Editor NEWOpen source btsync clone
Open source btsync clone
Very large JSON objects in the handshake could consume non-trivial amounts of memory and CPU on the server side.
Hi, I think you should display a donation option more prominently. I found a bitcoin address in the clearskies_core repo, but not many people go there, and there isn't even a bitcoin logo or a colorful donate button, or something to catch the eyes.
I think this project is pretty awesome, and I think many people might support the efforts if you made it a bit easier. Adding a paypal option might not hurt either.
The following test cases fail because there is no fork in Jruby.
Error:
test_0001_can_connect_and_send_data(GnuTLS::Session::acts as a client):
NotImplementedError: fork is not available on this platform
org/jruby/RubyKernel.java:1855:in fork' clearskies/test/gnutls.rb:16:in
run_test'
org/jruby/RubyBasicObject.java:1735:in `instance_eval'
Error:
test_0002_won_t_connect_with_wrong_password(GnuTLS::Session::acts as a client):
NotImplementedError: fork is not available on this platform
org/jruby/RubyKernel.java:1855:in fork' clearskies/test/gnutls.rb:16:in
run_test'
org/jruby/RubyBasicObject.java:1735:in `instance_eval'
Error:
test_0001_can_connect_and_send_data(GnuTLS::Session::acts as a server):
NotImplementedError: fork is not available on this platform
org/jruby/RubyKernel.java:1855:in fork' clearskies/test/gnutls.rb:16:in
run_test'
org/jruby/RubyBasicObject.java:1735:in `instance_eval'
Error:
test_0002_won_t_connect_with_wrong_password(GnuTLS::Session::acts as a server):
NotImplementedError: fork is not available on this platform
org/jruby/RubyKernel.java:1855:in fork' clearskies/test/gnutls.rb:16:in
run_test'
org/jruby/RubyBasicObject.java:1735:in `instance_eval'
I had a bit of trouble installing Clear Skies on my Debian Unstable and Raspbian distos.
The Ubuntu packages suggested don't seem to exist in Debian, so I tried to use the gem installation method, but that method requires the "ruby-dev" package.
Googling the problem shows that this is also true for Ubuntu, so I propose adding that to the instructions to simplify the installation process for others.
My abridged instructions:
[sudo] apt-get install ruby ruby-dev
[sudo] gem install rb-inotify ffi
The broadcast mechanism is not sufficient for networks which have different disjoint broadcast domains. The peer should report to the tracker it's list of known local addresses, so other peers in the same private network can readily find them, even their broadcast domains are different.
Size of chunks in the wire protocol be limited to a maximum to avoid DOS attacks on clients that reserve a buffer with the size of the chunk.
Hi
as agreed we should separate the protocol spec and implementation in different repositories.
If you run two clearskies instances you get a crashed thread, but clearskies continues to run. It should exit.
control> Thread crash: Daemon already running
control> /home/jake/repos/clearskies/lib/control_server.rb:23:in run' control> /home/jake/repos/clearskies/lib/control_server.rb:13:in
block in start'
control> /home/jake/repos/clearskies/lib/simple_thread.rb:72:in block (2 levels) in initialize' control> /home/jake/repos/clearskies/lib/simple_thread.rb:38:in
glock'
control> /home/jake/repos/clearskies/lib/simple_thread.rb:70:in `block in initialize'
The uTP code needs to try and open an IPv6 SharedUDPSocket, as well as as another STUNClient and UTPSocket.
Also, the tracker protocol needs to support a separate port number for IPv6. It almost certainly will be a different port number than the IPv4 port, especially since the IPv4 port is assigned by the NAT table on the router.
./clearskies detach ../aaaaa
clearskies/lib/control_client.rb:14:in []=': can't convert Symbol into Integer (TypeError) from clearskies/lib/control_client.rb:14:in
issue'
from ./clearskies:246:in `
Because of bitcoin miners, we now have dedicated devices that can hash SHA-256 in the order of Terahash/s. Since this rate is rising fast, SHA-256 will soon be obsolete.
I don't know enough to suggest any algorithm with confidance, but I'd also avoid s-crypt as dedicated devices for that algorithm are coming this year. S-crypt-n should be fine, however.
In core.md:
The server then uses the username to see if it has a corresponding access code or club. If it does, it completes the connection. The password or key is given to the TLS library as lowercase hexadecimal.
It isn't clear what is supposed to happen if it does not have a corresponding access code or club.
Looks like host B is trying to connect to an ipv6 port but host A only opened it for ipv4:
HOST A:
root@cereza:0:/home/piotr/devel/clearskies# lsof -Pni | grep 48555
ruby 19347 piotr 9u IPv4 1376349 0t0 TCP *:48555 (LISTEN)
HOST B:
tracker> New outgoing connection with A
tracker> Tracker discovered 10e53688a92370ced39427d8af929217b5445aa40c3cb69c890906231f8bbd33 b44ddbaf4d241e348568a515810654c0 utp A 46327
tracker> New outgoing connection with A
connection410> Opening socket to utp A 46327
connection410> Could not connect via utp to 2a02:8109:8440:8b8:f2de:f1ff:fe00:441b:46327: Address family not supported by protocol - sendto(2)
connection409> Opening socket to tcp A 48555
connection409> Could not connect via tcp to A:48555: Connection refused - connect(2)
I can look more into it tomorrow.
Could you add the required ports for opening up a firewall somewhere?
Hi there,
I just found a quite nice Linux GUI for Bittorrent Sync released under GPLv2, I think you guys might use parts of the code:
https://github.com/tuxpoldo/btsync-deb
Just in case you didn't know about it already.
See discussion at https://github.com/calmh/syncthing/issues/105
I'd like to take a stab at packaging this for Debian/Ubuntu if nobody is working on that.
Sounds good?
I was wondering how to get this section loaded on windows, I have tried installing gnutls (libgnutls26) with cygwin, have I missed anything?
https://github.com/jewel/clearskies/blob/master/lib/gnutls.rb
https://github.com/jewel/clearskies/blob/master/protocol/core.md#deleted-files explains the file deletion policy which is used to avoid ghost copies.
However, that policy has a flaw where a file which has been deleted cannot be re-added to that share.
Testing with the latest client reveals this problem (machines A and B):
Furthermore:
7. Copy the file manually to B
8. File is automatically deleted from A and B
In my view, this problem is as severe, if not worse than ghost copies, as it results in silent loss of data.
My proposal here would be to store the date of the file which was deleted along with the file name, and use it to decide whether any given file is a ghost or a new version.
On a sheevaplug running debian wheezy, the gnutls code segfaults. I have yet to determine if this is ARM specific.
"The ID and peer ID are combined into a single string separated by an "@" character, and sent as the "id" parameter."
It's not clear which ID is referring to. The ids should be clarified with an abbreviation consistent in the whole text.
./clearskies restart
./clearskies:149:in `
Commit 9c2f4 (Bind TCP to IPv6) broke one of my Clearskies setups: Raspberry Pi on IPv4-only network.
My other setup on a desktop in a dual IPv4+IPv6 network works fine, so I can't say if the problem is the architecture of the network.
The previous commit runs fine.
Here's the error log.
12:35:21.287118246 error main> Exiting due to exception in 'main' thread: Address family not supported by protocol - socket(2)
12:35:21.287557228 error main> /home/cachapa/clearskies/lib/network.rb:20:in initialize' 12:35:21.287898214 error main> /home/cachapa/clearskies/lib/network.rb:20:in
new'
12:35:21.288262200 error main> /home/cachapa/clearskies/lib/network.rb:20:in start' 12:35:21.288597186 error main> /home/cachapa/clearskies/lib/daemon.rb:62:in
run'
12:35:21.288930173 error main> /home/cachapa/clearskies/lib/daemon.rb:41:in daemonize' 12:35:21.289513149 error main> ./clearskies:147:in
The system version of rb-inotify keeps the global-vm-lock in older versions of Ubuntu. This may be a problem with FFI instead of rb-inotify.
The symptom is that the test as ran from rake
locks up completely.
If big files change often recalculating the sha256 hash and resending the whole file will be very inefficient. A rolling checksum mechanism like the one rsync uses would be much more efficient in case a large file is being modified constantly, for example when having on sync a directory where torrents are being downloaded.
It's a bit of a hassle to assume that all peers need to have the clocks synchronized.
$ ./clearskies debug
....
control_connection> Control error: incompatible character encodings: UTF-8 and ASCII-8BIT
control_connection> /home/badboy/code/clearskies/lib/access_code.rb:59:in `to_s'
/home/badboy/code/clearskies/lib/control_server.rb:115:in `handle_command'
/home/badboy/code/clearskies/lib/control_server.rb:59:in `block in serve'
/home/badboy/code/clearskies/lib/control_server.rb:53:in `loop'
/home/badboy/code/clearskies/lib/control_server.rb:53:in `serve'
/home/badboy/code/clearskies/lib/control_server.rb:40:in `block (2 levels) in run'
/home/badboy/code/clearskies/lib/simple_thread.rb:72:in `block (2 levels) in initialize'
/home/badboy/code/clearskies/lib/simple_thread.rb:38:in `glock'
/home/badboy/code/clearskies/lib/simple_thread.rb:70:in `block in initialize'
$ ./clearskies share $(pwd)/shared read-write
/home/badboy/code/clearskies/lib/control_client.rb:24:in `issue': server can't :create_access_code, says: Encoding::CompatibilityError incompatible character encodings: UTF-8 and ASCII-8BIT (RuntimeError)
from ./clearskies:143:in `<main>'
In Ruby 2.0 the default encoding is UTF-8 for all files (if not specified differently). This way access_code fails expecting ASCII-8BIT where it is UTF-8.
You should specifiy that you're handling binary data.
Is the protocol being worked on any more? I note that clearskies_core is now abandoned.
The Password section starts with "Instead of generating the master key". It might be clearer to say "Instead of generating a random read-write PSK". (I don't think the term "master key" is used anywhere else?)
Second, "The SHA256 algorithm is applied twenty million times to generate the 256-bit encryption key". Does this mean the read-write PSK? Is the PSK actually 256-bits in the passphrase case, but 128-bits for random keys?
Also, I was thinking it might be good practice for users to include a unique(ish) identifier as part of their passphrase, such as an email or domain name, in order to avoid accidentally colliding with someone else's passphrase.
This is looking pretty cool -- keep it up!
-greg
Is there any documentation on how to use the cli? I can't seem to get the daemon to start. Every time I try to run any of the commands I get 'Daemon not running'. I ran the tests and I get:
test_0004_should detect deleted files(Scanner::finds files)
test_0002_should detect new files(Scanner::finds files)
I am not sure what to do. I am not too familiar with ruby but I would still like to contribute.
Hello.
While reading the protocol draft, I didn't catch the following idea. When a user want to give full access to a share to a peer, does he have to generate a temporary access code or it's enough to share only a read-write secret (like in BT Sync)?
Thanks in advance.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.