inconshreveable / go-update Goto Github PK

I think you mentioned that this was still a problem for you. We've got the same problem in hk at the moment since we can't delete a running executable on Windows.

Just curious how you were planning on solving this one (and if this is the right place for it, or if you consider that ngrok's problem to solve).

go-update should allow you to update a file that doesn't exist on the host system yet. (cc: @lukechampine )

Hi there.. An security audit of a software using this package revealed a possible problem with this package. I think that this is relevant also for upstream even If I can (and am) fixing this for my own usage as well.

Here's a a slightly edited section from the report, I have added the ED25519 hash myself:

Updater allows for a rogue Version Downgrade (Medium)

Update metadata (including a SHA256 hash over the updated binary and an ED25519
signature over the SHA256 hash

...

After that a patch dedicated to updating to the new version is downloaded over
HTTPS.

...

The code in pkg/upgradebin/upgradebin.go then uses vendor/github.com/
inconshrevable/go-update /apply.go to verify and apply the update. Within this
process the patch is firstly applied to the current binary in order to create a
new binary. Before the old binary is replaced with the new one, the SHA256
checksum of the new binary is compared with the expected one as well as checked
against the received ED25519 signature. Besides the usage of TLS as transport
and the security provided by ..., there are no cryptographically
protected version checks occurring throughout this process. Therefore it opens
doors for an attacker without the ED25519 update signing key who nevertheless
either has access to [server] or is otherwise in possession of a valid TLS
certificate for [server] and MitM access 5 somewhere between the bridge and
[server]. In effect, the attacker could deliver a malicious “update” to a
victim, causing the binary to be downgraded to an earlier less- secure version.
This could mean that if the victim has version 3 and the attacker wants to
rollback to version 2, he requests the update metadata from version 1 to
version 2 from the server, obtaining a signature over version 2. He or she
would then need to calculate a patch from version 3 to version 2 and deliver it
to the victim together with the obtained signature.

It is recommended to amend the updater so that the signature is calculated
over the hash of the concatenation of the target version, a delimiter that
cannot occur in a version number (e.g. a null byte) and the binary (or its
hash). The updater should additionally verify that the target version is in
fact higher than the current version.

On windows i noticed that McAfee virus scanner deletes the new exe after update and give notification of possible malware. Anyway around that without disabling the virus scanner?

Wouldn't it be cool if go-update also supported automatic reload of the go binary (itself)? I don't know if it's possible or even desirable in the long run it would make the deploy process even smoother, just push a new release and then all nodes running a service would auto update and restart!

Are we required to use your service to use this library or can we stand up the update service on our own?

Starting from scratch with go-update can be time consuming for someone without much domain knowledge. The documentation could be improved in the following areas:

• How to generate the required certificates & which formats (i.e. elliptic curve prime 256)
• A walkthrough of a end to end experience (generating certs, signature, hash, diff)
• Clean up of the language used regarding which object the signature should be generated from (currently seems ambiguous)

I have started to document the process by creating this gist.

During your talk, there was a panic when you tried to update (it looked like a nil pointer was being dereferenced somewhere)...

I'm just curious here, but once you figure it out, can you post what it was?

Nice talk!

can somebody write any example handler for update?

func updateHandler(w http.ResponseWriter, r *http.Request){

file, err := os.Open("myprogram.exe")

etc....

}

Updating (at least under linux) changes the permissions on the file.
Before the update i set ther permissions to 755.
After the update the file has 700.

With this updating a file for other users (updating a file in /usr/bin as root) makes the file unusable for any other user.

There seems to be some code to set permissions to 755 in apply.go, but it seems to be not working.

Equinox will shut down on September 30th, 2021. After that date, all accounts and applications will be deleted. Creating new accounts and new applications will be disabled shortly.

The documentation mentioning Equinox should be removed:
/README.md
/doc.go

Is there a method to indicate the progress of the download？

Currently, the library does not resolve possible conflicts when performing a self-update in parallel processes (and probably should not). The user might face the following errors:

# unix: attempt to update current program path that points to non-existent file after the parallel self-update
Error: rename /home/user/go/bin/.binary.old /home/user/go/bin/..binary.old.old: no such file or directory

# windows: attempt to replace the real current binary file (.binary.exe.old) with the result binary file (binary.exe) from the parallel self-update
Error: rename C:\Users\user\go\bin\binary.exe C:\Users\user\go\bin\.binary.exe.old: Access is denied.

To avoid such issues, locking together with a delay between updates can be used (example here).

go-update is currently targeted at only updating a single file. And while it's possible to call it repeatedly to update many files, it makes more sense for an archive of many files that compromise an update to be checksummed and verified all at once. Come up with an API or easy example on how to accomplish this.
cc: @lukechampine

I get this error:

rename C:\Users\PB\Desktop\bazooka.exe C:\Users\PB\Desktop\.bazooka.exe.old: The process cannot access the file because it is being used by another process.

which seems to be produced at this line: https://github.com/inconshreveable/go-update/blob/master/update.go#L374

The process is under Windows, running as administrator. Any clues ?

The current mechanism is not robust in two ways. It can be fixed completely on POSIX and be fixed somewhat on Windows.

1. This is also documented in your code: The old file is moved to a backup path, then the new file is moved from a temp path to the target path. If the world stops between these steps, there is no file at the target path. At least POSIX guarantees that a file rename(2) is atomic, i.e. if you do not move the old file from the target path, but "over"-os.(*File).Rename() the new file to the target path, then whenever the world stops, there will always be either the old file or the new file. Never no file. This atomic rename guarantee is valid on POSIX, only valid-ish on Windows.

2. You should fsync(2) the files. When the world stops you do not have any guarantee of what your files look like, except for the backup file. If Go's file methods resemble the POSIX semantics (I assume they do), then os.(*File).Close() is not enough. You need to do os.(*File).Sync() before os.(*File).Close(). Crashing a process and crashing the machine are two different error modes. Writing files, then not syncing the files, then renaming files, then pulling the power cord leads to unpredictable results even with journaling filesystems.

I'd suggest to do the following:

1. copy from target path to backup path
2. sync then close backup file
3. write new file
4. sync then close new file
5. rename new file to target path
6. on rollback rename backup path to target path.
7. on success hide or delete backup path

If possible do a hard link instead of a copy in step 1 (would need admin privileges on Windows, IIRC).

There were quite extensive and enlightening discussions about filesystem consistency myths back in the day. For some background information refer to Ted Ts'os blog post Don't fear the fsync.

/e fixed web link

Hello, is this functionallity already present? I just looked into the code and found that it only replaces the current file, but does not kill that and start the new file. Do I need to make my own implementation to control the execution?

Regards
Jorge Luna

PSA: It appears that the author of this project is no longer maintaining it. https://github.com/doitdistributed/go-update seems like a fork that is more up-to-date.

Thank you for this library! It has been working well for me/us for 3+ years

If you have a file you are going to go-update, and you run
sudo chown $DIR
Then run the code that runs update.Apply
You get failed to apply update.

If you then chown the directory, the update application will work.

To fix this, I think we need to add a step to verify the permissions of the parent directory for the target file before applying the update and return an informative error.

I just whipped up a first draft at the 2.0 API. Take a look: https://github.com/inconshreveable/go-update/tree/wip I think it's much cleaner, but I could use feedback from anyone who has a vested interest in go-update.

The goals of the 2.0 redesign is as follows:

1. go-update should expose the smallest possible API that is still immediately useful.
2. Remove the subpackages. How an application decides what to update from is out of scope. How to generate updates is out of scope. How to download updates is out of scope.
3. Remove the "fluent" API in favor of a more idiomatic Go API with a single Update type with public properties the caller can set to configure the update process to their application's needs.
4. go-update should make no assumptions about the implementation details of the patching, checksumming, or signature verification pieces. All of these must be pluggable.
5. go-update should allow the update to create entirely new files that didn't exist before
6. (Maybe) go-update should allow updates from an archive

Here's a quick example of what it looks like to use the library now:

import (
    "crypto"
    "fmt"

    "github.com/inconshreveable/go-update"
)

func doUpdate(url string, signature []byte, checksum []byte, publicKeyPEM []byte) error {
    up := Update{
        TargetPath: "/path/to/target_file",
        TargetMode: 0777,
        Checksum:   checksum,
        Signature:  signature,
        Patcher:    update.BSDiffPatcher,
        Verifier:   update.ECDSAVerifier,
        Hash:       crypto.SHA256,
    }   
    if err := up.FromURL(url); err != nil {
        return err 
    }   
    if err := up.VerifyWithPEM(publicKeyPEM); err != nil {
        return err 
    }   
    err, errRecover := up.Do()
    if err != nil {
        if errRecover != nil {
            panic(fmt.Sprintf("update failed to recover from failure: %v. Original error: %v", err, errRecover))
        }   
        return err 
    }   
    return nil 
}

This is a sketch of the API, but I like its minimalism and I think it's a good place to iterate from. It compiles, but I haven't wired up any of the tests yet. I'll do that other next days as we iterate on API design.

@lukechampine: Thoughts?
@titanous: Thoughts? Would this be sufficiently general to be useful for go-tuf?

For...

func doUpdate(url string) error {
	resp, err := http.Get(url)
	if err != nil {
		return err
	}
	defer resp.Body.Close()
	err := update.Apply(resp.Body, update.Options{})
	if err != nil {
		// error handling
	}
	return err
}

I get: no new variables on left side of :=

I'm currently packaging your library in Debian, as a dependency of several other packages.

You currently do not tag any releases, and this makes it harder for downstream packagers.
(Also, referring to specific commits is awkward)

@inconshreveable I think you said you wanted to keep this discussion going in an issue here.

I opened a PR for osext to fix the darwin symlink issue: https://bitbucket.org/kardianos/osext/pull-request/1/eval-symlinks-on-darwin-to-get-true

Once that's done, it should be good enough to replace the usage of go-execpath in this project, and that should clear the way for hk to start using go-update.

Syscall.Umask should be only for unix systems

Hi,
We would like to use the executable path detection logic in our GO code. Do you mind exposing osext as an independent package?

Thanks.

Just curious if this will work if your application is wrapped inside of a dmg application for macos.