psanford / wormhole-william Goto Github PK
View Code? Open in Web Editor NEWEnd-to-end encrypted file transfer. A magic wormhole CLI and API in Go (golang).
License: MIT License
End-to-end encrypted file transfer. A magic wormhole CLI and API in Go (golang).
License: MIT License
sorry, wrong repo
I took a look through the code, cli, and existing examples but i couldn't seem to find anything explaining how to host your own rendezvous server
Bring this over from:
#26 (comment)
interface is inconsistent about units of measurement: kB and KiB
Receiving file (14.0 kB) into: Von Franz Dream Analysis v. 0.3.txt
ok? (y/N):y
13.66 KiB / 13.66 KiB [---------------------------------------------
In Least Authority, we have recently two apps based on wormhole-william and magic-wormhole protocols: Destiny and Winden.app. We have made some changes in our fork of wormhole-william during development (C bindings, WASM bindings, Security audit findings implemented, Hashcash, Websocket implementation, and various smaller fixes/ refactoring). We would like to upstream them and make it easier for other builders to use these changes.
[Differences] (master...LeastAuthority:wormhole-william:master)
There are few questions pop out:
We have some further plans to continue development and add more features, for example, dilation and multiple files sending support. We would be happy to contribute these changes too and even help with maintaining the upstream repository to boost usage.
Why is it named wormhole william? Can you add a FAQ?
Hi! I'm planning on distributing the binaries from this project (with attribution) on npm, the package manager for JavaScript. This is a very convenient way for JS devs who already have node installed to use this project - it is just be a matter of running npx magic-wormhole
, no manual downloads necessary.
(Yes, it's a little scary that everyone is so comfortable with this, but leave that aside.)
I wanted to give a heads up here in case you'd like me to take it down, or alternatively if you'd like to make this official, in which case I would be very happy to send a PR which adds a GitHub action which automatically publishes the new binaries whenever a GitHub release is created.
This is not the same as making it possible to use this project from JavaScript; that's more work (which I started here but never got back to). It'll just be a convenient way of distributing the binaries.
I successfully built wormhole-william on AMD64, but it fails on my Raspberry Pi.
go version go1.13 linux/arm
wormhole/file_transport.go:140:28: constant 4294967295 overflows int```
The latest release is v1.0.6 from December 2021. That will soon be two years ago. A lot has happened since then on the master branch and I have various open PRs at https://github.com/psanford/wormhole-william/pulls that are waiting for reviews and merge.
It is time to do a new release. Not only does a new release mean rebuilt cmd binaries with all of the security fixes from the new Go compiler releases (assuming #93 is merged) but also that projects like rymdport get access to all of the features since the last release.
The examples directory has examples for everything except for receiving directories. Adding that example would make the examples directory complete for anyone wanting to see how the API can be used.
Not quite done yet.
I was looking at comparing the .CompressedBytes64
and .TransferBytes64
of *IncommingMessage
to compare how much the compression managed to save. However, what I found was that they both were zero for text sends. The same thing occurred with the same variables without the 64 postfix. I can understand if the text perhaps isn't compressed and that it then would be equal to the transfer bytes but it would be good if the transfer bytes would show how many bytes of text there are.
My use case for this is to have a bytes.Buffer
that I grow the the amount of bytes received in order to be able to not append to the buffer each time and thus improve performance. It turns out that I have been growing it with zero each time, for quite a long time...
Hi there. I am wondering if the need for supplying an io.ReadSeeker
possibly could be changed to using a regular io.Reader
?
The question comes from work on wormhole-gui where I would like to try and get it to work on iOS and Android. Getting it working would currently result in a lot of platform specific code for getting a seeker working or might even be impossible when working through the limited APIs exposed by Apple.
It looks like there isn't a massive amount of code relying on it (see search results) and hopefully those size calculations could be done in a different maner. Just using ìo.Reader
would make things much better for mobile and wouldn't cause any API breakages as any ìo.ReadSeeker
already complies with ìo.Reader
.
Thank you for the project. I use it regularly on multiple machines. But presently the linux builds require a specific libc runtime which is not available in my target hosts. So I have to build with CGO_ENABLED=1 CC=musl-gcc go build --ldflags '-linkmode external -extldflags=-static'
and use it.
It would be super helpful for people like me if you can also release linux builds with musl libc.
I really appreciate the existing command completion!
IMHO it would be even cooler to have a completion for the wormhole code. It could just look up the word in the dictionary, for example.
See magic-wormhole/magic-wormhole.rs#136 for details.
We can omitempty
these fields.
It would be nice to leverage some type of .rc
file for wormwhole-william
. Especially for those who have a different relay url
which is the case for myself.
This is still open on the python wormhole project as well: magic-wormhole/magic-wormhole#338 (environment variable and config file are discussed here) So it might be worth aligning it so that both clients can use the same config.
I don't see why not provide these too. Besides, Windows 11 is 64-bit only.
Sending text does not seem to update WithProgress SendOption if it is provided. This leads to progress bars not updating like they should. This was initially reported Jacalz/rymdport#2 and later fixed with Jacalz/rymdport@ae4d314 by manually updating progress on my side.
I have now replicated the same issue in a test application that I wrote to make sure that the issue wasn't in my application.
When this issue is resolved we should upgrade nhooyr.io/websocket to make dependabot not warn about this dependency: coder/websocket#318
This is a test only dependency so its not an actual issue for us. But it would be nice to clear the dependabot issue.
Would it be possible to use ANSI escape codes to set the colors of the characters to black and white when printing the QR codes?
I decided to run gosec on the whole code base by running ~/go/bin/gosec ./...
and the result was a total of 55 security issues that ranged between low and medium severity.
I attach the html
formatted security report (as a zip file due to GitHub restrictions) here so it can easily be opened in the browser to sort through the security issues.
security-report.zip
If peer sends a relay server in their transit hints, that hint should be followed. Otherwise, clients with different configured relay servers cannot find each other over a relay connection. Conversely, the own configured relay server should be sent over as hint (not sure if this is done currently).
Does wormhole-william
supports listening on the receiving side for direct-tcp-v1
connections? By looking at the code it seems that direct-tcp-v1
are supported only when wormhole-william
is sending a file
wormhole-william/wormhole/wormhole.go
Lines 270 to 283 in 1a5ce30
The hostname
, port
and priority
fields on transitHintsV1
are wrong* and should be removed. All relays should be encoded using transitHintsV1Hint
instead.
* respective to the current Python implementation as source of truth. It is totally possible that this once was an acceptable serialization format some time ago.
Hey, this is awesome! I'm totally floored that you wrote this.
One thing that might help me manage the server better.. could you add a client_version
field to the BIND message that shows the connections as coming from the Go implementation? Probably at
wormhole-william/rendezvous/client.go
Lines 514 to 518 in 6689c61
client_version
tuple to ("golang-william", "v1.0.0")
(or however you've got the released tagged.. maybe use the output of git describe --tags --always
if that's convenient, that's what the python version does when you build out of a git checkout).
I've been meaning to set up a monthly "state of the wormhole" email, and I'll include the server's counts of client versions in there. Right now I see 228 connections in the last 24 hours that don't report a client_version
, but those might also be python clients from before 0.10.5
(released Feb 2019) when I added the reporting feature.
thanks!
Upon installation, IMHO it would make sense to enable the shell completions by default for any of bash, zsh and fish present.
I'm trying to run the arm binary you release under the name "arm7" on the raspberry pi 1.2 .
Downloaded from here, https://github.com/psanford/wormhole-william/releases/tag/v1.0.4
$ chmod + ./wormhole-william-linux-arm7
$ ./wormhole-william-linux-arm7
-bash: ./wormhole-william-linux-arm7: cannot execute binary file: Exec format error
$ file wormhole-william-linux-arm7
wormhole-william-linux-arm7: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, Go BuildID=0YIZepyXHTYAuc-8uFmp/jzXcGXEBLXzkuLditY_u/UT2EVchW_bDyh8MKe3ZE/fRE1lMfbIQn6xOdolUqB, not stripped
As you can see this is different from what the vast majority of arm binaries for this platform (should be called "armv7h" ) give. Example, https://github.com/Jguer/yay/releases/tag/v10.0.4
$ file yay_10.0.4_armv7h/yay
yay_10.0.4_armv7h/yay: ELF 32-bit LSB pie executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, BuildID[sha1]=0dcad9ae91e4d262acb63017783c5938ced94e90, for GNU/Linux 3.2.0, stripped
That "x86" shouldnt be there. Maybe something got mixed up in the build process ?
PS: I dont have an arm 64 computer to test run it (Raspberry Pi 4), but I also checked the one you release as "arm64". Should be called "aarch64".
$ file wormhole-william-linux-arm64
wormhole-william-linux-arm64: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, Go BuildID=EFhjyyM10MGZsAiY14oG/kVXqSI1C0fHbXGObwjtk/1GAEHHv7bpwD2U-kOm1Q/q7gQMrO0gj6zdscFn8PP, not stripped
This is how an arm binary for 64bit platform should look like
$ file yay_10.0.4_aarch64/yay
yay_10.0.4_aarch64/yay: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, BuildID[sha1]=149f0144183ad856fe2b7192bf1744fd622d032d, for GNU/Linux 3.7.0, stripped
Same thing with the arm5 and arm6
I tried to run that in Termux on my phone, but no luck.
Is it due to the limitation of Android 10 operating system or by some other reason?
Hope to get any help!
~ $ ./wormhole-william-linux-arm64 send get-pip.py
Error sending message: dial ws://relay.magic-wormhole.io:4000/v1: failed to WebSocket dial: failed to send handshak
e request: Get "http://relay.magic-wormhole.io:4000/v1": dial tcp: lookup relay.magic-wormhole.io on [::1]:53: read
udp [::1]:47274->[::1]:53: read: connection refused
First of all thank you for your work in this project and especially the thought of releasing the prebuilt binaries
I suppose you have or are using some kind of automated infrastructure or service to do this, so I think this request wouldn't be too much of an extra effort.
But it would be extremely usefull, for getting wormhole on tiny less powerfull computers , SBC boards etc. Think of Raspberrry Pi and alike. It's not practical to download the entire Go chain infrastructure and build on them.
Right now, you do release an arm64. But this only supports the most recent Raspberry Pi's for example. It leaves out an enormous amount of previous devices.
I also know it can be cumbersome to support "all" of arm architectures . Should it be arm5 or armv6 or ... ?
But I've seen from other (go) projects that do this, that usually all it takes is one single arm (not 64 bit) binary for all of them. Go projects 99% of the time run wonderfully on all them.
So I would like to request if you could provide this.
If a "universal" not 64 bit binary not possible, then at least an armv7h binary for the Raspberry Pi 2.
Thanks in advance
Hi this is just a question.
Using wormhole-william on Apple Silicon with GUI client.
I would like to know whether your implementation of the wormhole protocol already includes the ability to queue files or not ?
F.ex. when sharing multiple individual files and sharing their individual codes to the recipient, when the latter launches multiple shares at once, will they all start being streamed ?
If so, can i suggest to implement the ability to queue the shares, so that when the recipient launches them all, only the first one launches (to benefit of all the bandwith) and only starts the next one, when the first one is finished.
Would this be possible ?
The python implementation offers optional support for transport via tor. Is there an intention to add this to wormhole-william?
As of 049df45, cancelling incomming and outgoing connections works great. However, it would be nice if the errors could be a bit more graceful. This does not affect the latest stable release yet.
Cancelling the outgoing send returns this (first error being in the sending end and the second in the receiving end):
2021/05/20 21:01:28 Fyne error: Error on sending file
2021/05/20 21:01:28 Cause: write tcp 192.168.1.220:43901->192.168.1.220:36580: use of closed network connection
2021/05/20 21:01:28 At: /home/jacob/go/src/github.com/Jacalz/wormhole-gui/internal/transport/bridge/send.go:112
2021/05/20 21:01:28 Fyne error: Error on copying contents to file
2021/05/20 21:01:28 Cause: unexpected EOF
2021/05/20 21:01:28 At: /home/jacob/go/src/github.com/Jacalz/wormhole-gui/internal/transport/receiver.go:71
Cancelling the incoming receive returns this. Here, the first error (from the receiving end) is what I would like to see, but the second error is not quite as graceful.
2021/05/20 21:02:02 Fyne error: Error on copying contents to file
2021/05/20 21:02:02 Cause: context canceled
2021/05/20 21:02:02 At: /home/jacob/go/src/github.com/Jacalz/wormhole-gui/internal/transport/receiver.go:71
2021/05/20 21:02:02 Fyne error: Error on sending file
2021/05/20 21:02:02 Cause: write tcp 192.168.1.220:43167->192.168.1.220:55670: write: connection reset by peer
2021/05/20 21:02:02 At: /home/jacob/go/src/github.com/Jacalz/wormhole-gui/internal/transport/bridge/send.go:112
I would like it if all the errors could fail gracefully with "context cancelled".
Hi,
Running govuln, I get:
Found 1 vulnerability in packages that you import, but there are no call
stacks leading to the use of this vulnerability. You may not need to
take any action. See https://pkg.go.dev/golang.org/x/vuln/cmd/govulncheck
for details.
Vulnerability #1: GO-2022-0493
When called with a non-zero flags parameter, the Faccessat
function can incorrectly report that a file is accessible.
More info: https://pkg.go.dev/vuln/GO-2022-0493
Found in: golang.org/x/[email protected]
Fixed in: golang.org/x/[email protected]
I don't have a go toolchain installed, so I basically can't use this, as far as I can tell. It would be nice if there were built packages.
I'd like the easier deploy for wormhole-william but what I use "magic-wormhole" for is "wormhole ssh invite" and "accept".
This small feature is really great timesaver.
wormhole ssh --help
Usage: wormhole ssh [OPTIONS] COMMAND [ARGS]...
Facilitate sending/receiving SSH public keys
Options:
--help Show this message and exit.
Commands:
accept Send your SSH public-key In response to a 'wormhole ssh invite'...
invite Add a public-key to a ~/.ssh/authorized_keys file
As reported in magic-wormhole/magic-wormhole.rs#160 (comment), Wormhole William is also affected. For the Rust<->Go and the Python->Go hang I suspect it is that the Go implementation expects an empty record, which the other implementations simply skip. At least this is where I'm looking for errors at the moment.
I'm really glad to have found a Go version of Magic Wormhole! This makes installation a breeze by being able to simply download a single binary from my private server.
Small issue so far though while trying it out: With Magic Wormhole I can do
wormhole send --text "hello"
or even piping, like
ls -1 | wormhole send --text -
but neither of these seem to work with wormhole-william
. Am I missing something or if not, are these features in the roadmap?
The reflect
package in Go is slow as it has to do runtime reflection on the types. This also has the added negative effect that binaries generally get bigger due to having to embed more type data than what otherwise would have been necessary. Using reflect is generally something that, like the unsafe
package, should be avoided where possible.
I have looked at the code and some of the usages should be possible to remove by creating interfaces with getters and setters instead.
For the main user-facing API, the usages are in these files:
For testing:
Removing the dependency on reflect should result in smaller binaries with generally big performance improvements for the parts where reflect was being used.
If possible, it would be great if the receiver never needs to specify -v/--verify, and the verification string would automatically be shown when the sender required it. (And when the sender did not ask for it, the string is never necessary for the receiver, even though he would get one if wormhole-william recv -v
was given).
I couldn't find wormhole on any of the popular package managers on mac so I had to compile it myself and add it to /usr/local/bin manually. It'd be great if this was on macports and homebrew so people could more easily install it and more importantly, keep it up to date. Especially since the python implementation isn't distributed there either.
Is this something that anyone would like to look at? Otherwise I'd by happy to do it.
I am using wormhole-william
for my own project and there it serves as a way of syncing data between two devices. I was trying to set a timeout because I don't users of my application to have the sending running forever if they start it. Right now the only way to stop it is to receive on a different application or kill the application.
Context values are already part of the function parameters that need to be used when calling the function, so the API would not need to change to accommodate this.
I would for example like to have this working:
package main
import (
"context"
"fmt"
"log"
"os"
"time"
"github.com/psanford/wormhole-william/wormhole"
)
func main() {
if len(os.Args) < 2 {
fmt.Fprintf(os.Stderr, "usage: %s <file>\n", os.Args[0])
os.Exit(1)
}
filename := os.Args[1]
f, err := os.Open(filename)
if err != nil {
log.Fatal(err)
}
var c wormhole.Client
ctx, _ := context.WithTimeout(context.Background(), 10*time.Second)
code, status, err := c.SendFile(ctx, filename, f)
if err != nil {
log.Fatal(err)
}
fmt.Println("On the other computer, please run: wormhole receive")
fmt.Printf("Wormhole code is: %s\n", code)
s := <-status
if s.Error != nil {
log.Fatalf("Send error: %s", s.Error)
} else if s.OK {
fmt.Println("OK!")
} else {
log.Fatalf("Hmm not ok but also not error")
}
}
I would greatly appreciate seeing this added. It would be highly beneficial for my application and for a lot more people too if I guess. Having this implemented would also mean that context.WithDeadline()
should work and possibly context.Cancel()
, but I haven't tested that.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.