2.0 API about go-update

[titanous]

TUF handles the hashing, signature verification, and download parts. Here's what the API currrently looks like (slightly simplified for explanatory purposes):

type Client interface {
  // Update checks for updates and returns any added/changed targets.
  Update() (Files, error)
  // Targets returns a list of all available files.
  Targets() Files
  // Download downloads a named target into dest. The destination must
  // not be used until the download is complete.
  Download(string, io.Writer) error
}

type Files map[string]FileMeta

type FileMeta struct {
  Length int64
  Custom json.RawMessage
}

The Client is initialized with the root key(s) and remote store to download from (typically an HTTP downloader with a base URL).

Note that the details of the hash/signature verification are completely hidden from the user, as they are easy to screw up (there is quite a bit going on behind the scenes).

With that in mind, I'd suggest dropping the Checksum, Signature, Verifier, and Hash fields, as well as the VerifyWithPEM method, as they are all too low-level to be used safely. After doing this, the update flow looks like:

1. Initialize TUF client, and call Update.
2. Figure out what target (if any) we want (perhaps go-update can help here).
3. Create a temporary file and Download into it.
4. Tell go-update to perform the update using the temporary file.

[inconshreveable]

Okay, sounds like I'll need to read TUF and understand all the attack vectors before continuing here. I'm reticent to remove those fields and functionality because I want go-update to support those operations (as safely as possible) so that it's a good primitive to work with even if you're not using TUF.

It should obviously be flexible enough to turn those things off if you're handling them at a higher level (like in TUF).

[titanous]

Here's the list of attack scenarios: https://github.com/theupdateframework/tuf/blob/v0.9.9/docs/tuf-spec.txt#L121-L155 (though the whole spec is worth a read)

[inconshreveable]

Awesome, thanks! Definitely reading the whole spec.

[lukechampine]

I agree with @titanous in that the Update struct feels a bit field-heavy. Instead of removing fields, though, I'd rather have optional fields with sane defaults. For example, if TargetMode is not set, go-update should default to reusing the existing mode bits. (Admittedly, this breaks for "new" update files -- maybe require explicit mode bits in that case?)

I would also remove the Verifier and Hash fields and instead make them arguments to the Verify functions. And having a Do() function makes me a bit sad. Maybe I'm misreading this, but it seems like the Update struct is meant to be reused? In that case, your design decisions make more sense, but personally I don't envision myself reusing an Update.

go-update should make no assumptions about the implementation details of the patching, checksumming, or signature verification pieces. All of these must be pluggable.

I think this is the right approach, though I'm wary of how it might impact ease-of-use. The log package is a good example of how you can provide both low-level controls and friendly defaults.

[inconshreveable]

The Update structure is intended to be used once and not reused. I was modeling the design after something like http.Client: http://golang.org/pkg/net/http/#Client and http.Server: http://golang.org/pkg/net/http/#Server

The docs will help make it clear that every field is optional and has a sane default except the Update io.Reader. The minimal use of the package would look something like this:

func updateTo(newbytes []byte) {
    up := update.Update{Update: bytes.NewReader(newbytes)}
    err, errRecover := up.Do()
}

Perhaps a better design would be to make the single required parameter part of the call to Do and leave the rest of the fields on the structure as optional:

func updateTo(newbytes []byte) {
    err, errRecover := new(update.Update).Do(bytes.NewReader(newbytes))
}

Another possibility would be to design it as a top-level package function Do or maybe Update and rename the existing structure into Options or Config.

func updateTo(newbytes []byte) {
    err, errRecover := update.Update(bytes.NewReader(newbytes), update.Options{
        // all optional fields go here
        TargetPath: "/path/to/file",
    })
}

I'm not sure I understand what you mean about the changes to Verify

[lukechampine]

Good to see we're mostly on the same page. As for Verify, I just meant that updates would look like:

up := Update{
    TargetPath: "/path/to/target_file"
}
if err := up.VerifyChecksum(checksum, crypto.SHA256); err != nil {
    return err
}

instead of:

up := Update{
    TargetPath: "/path/to/target_file",
    Checksum:   checksum,
    Hash:       crypto.SHA256,
}
if err := up.VerifyChecksum(); err != nil {
    return err
}

Now that I rethink it though, I'm not so sure which is better. Managing all these optional fields is tricky. The current method of chaining functions feels pretty clean, in part because you can almost read the call chain like an sentence: "update (and) VerifyChecksum FromUrl." However, specifying everything beforehand simplifies error handling, because you only have to check the errors returned by update.Do (or update.Update, or maybe update.Execute or update.Apply).

[inconshreveable]

I like update.Apply quite a bit.

[titanous]

We're rolling out our TUF-based updater with go-update for the Flynn CLI, the implementation might be useful for reference: https://github.com/flynn/flynn/blob/5fd0d5fac00f1baec4124e602b88a3e030e8fe49/cli/update.go#L73-L140

[inconshreveable]

merged to master, closing