softprops / shiplift

🐳 🦀 rust interface for maneuvering docker containers

shiplift

a rust interface for maneuvering docker containers

install

Add the following to your Cargo.toml file

[dependencies]
shiplift = "0.7"

usage

Many small runnable example programs can be found in this repository's examples directory.

Doug Tangren (softprops) 2015-2018

shiplift's Issues

support docker pull/push from private registry?

Is there any plan to support docker pull/push from private registry?
It should add a header in the http request, But it is no easy to do so, it needs lots of changes.

release master to crates.io

Not a big deal for me because cargo's git support is great, but users don't want to have to go to github and figure out if they can use master or not.

Return "Iterator<Result<Event>>" instead of "Result<Iterator<Event>>" for Docker::events()

If there's an error in initialization, it can be returned as the first (& only) element of the iterator. The current behaviour doesn't propagate any errors that occur while iterating the stream, it panics instead.

Move away from kcov for codecov

Historically it's been a pain to get kcov working correctly and accurately with rust as seen with the current Travis config. I've had slightly better experiences with the tarpaulin crate. Let's migrate towards that as a most sustainable solution

Exposed ports

Hello, is it possible to add ports to expose when creating a docker container?

streaming interface

SSL-related build failures with 0.3.1

I get this:

cargo:warning=src/c_helpers.c: In function ‘rust_SSL_clone’:
cargo:warning=src/c_helpers.c:4:5: warning: implicit declaration of function ‘CRYPTO_add’ [-Wimplicit-function-declaration]
cargo:warning=     CRYPTO_add(&ssl->references, 1, CRYPTO_LOCK_SSL);
cargo:warning=     ^~~~~~~~~~
cargo:warning=src/c_helpers.c:4:20: error: dereferencing pointer to incomplete type ‘SSL {aka struct ssl_st}’
cargo:warning=     CRYPTO_add(&ssl->references, 1, CRYPTO_LOCK_SSL);
cargo:warning=                    ^~
cargo:warning=src/c_helpers.c:4:37: error: ‘CRYPTO_LOCK_SSL’ undeclared (first use in this function)
cargo:warning=     CRYPTO_add(&ssl->references, 1, CRYPTO_LOCK_SSL);
cargo:warning=                                     ^~~~~~~~~~~~~~~
cargo:warning=src/c_helpers.c:4:37: note: each undeclared identifier is reported only once for each function it appears in
cargo:warning=src/c_helpers.c: In function ‘rust_SSL_CTX_clone’:
cargo:warning=src/c_helpers.c:8:20: error: dereferencing pointer to incomplete type ‘SSL_CTX {aka struct ssl_ctx_st}’
cargo:warning=     CRYPTO_add(&ctx->references,1,CRYPTO_LOCK_SSL_CTX);
cargo:warning=                    ^~
cargo:warning=src/c_helpers.c:8:35: error: ‘CRYPTO_LOCK_SSL_CTX’ undeclared (first use in this function)
cargo:warning=     CRYPTO_add(&ctx->references,1,CRYPTO_LOCK_SSL_CTX);
cargo:warning=                                   ^~~~~~~~~~~~~~~~~~~
cargo:warning=src/c_helpers.c: In function ‘rust_X509_clone’:
cargo:warning=src/c_helpers.c:12:21: error: dereferencing pointer to incomplete type ‘X509 {aka struct x509_st}’
cargo:warning=     CRYPTO_add(&x509->references,1,CRYPTO_LOCK_X509);
cargo:warning=                     ^~
cargo:warning=src/c_helpers.c:12:36: error: ‘CRYPTO_LOCK_X509’ undeclared (first use in this function)
cargo:warning=     CRYPTO_add(&x509->references,1,CRYPTO_LOCK_X509);
cargo:warning=                                    ^~~~~~~~~~~~~~~~
ExitStatus(ExitStatus(256))

13ef409 fixes this.

exec should return output

The return type for calling exec is Result<()>, meaning it doesn't return any output at all.

Panic when building a new image

When running docker.images().build(...), it ends up panicking with this message:

expected build output stream or error

This is because one of the JSON objects is not a "stream", "status" or "error":

DEBUG:shiplift: Object({"aux": Object({"ID": String("sha256:3fdcaf0ab1b77ae77395563c779592fa6c00b210c38d5f83c6b01d5da2ccf2df")})})

Instead of identifying and repackaging each object, would it be better to simply return the JSON iterator? As far as I can tell, there isn't any value added by trying to repackage the objects into native structures. I also can't find a spec for the response JSON, so this issue might keep popping up over time. Manual doesn't help: https://docs.docker.com/engine/api/v1.31/#operation/ImageBuild

let API names follow Rust idiom

I saw two examples so far:

• rep::Event::timeNano should be rep::Event::time_nano
• errors::Error:IO(IoError) should be errors::Error:Io(IoError)

Const faults

These do not vary based in ctx. They should be constantized

New release on crates.io

The last few weeks got us some amazing new features, especially network support. When can we expect a new release on crates.io?

create issue templates for feature requests and defects

separate what's not working as expected from what we'd like to add new support for

Bump version

Hey, could you please bump the version? It's been a while an there are some critical fixes.

Port to serde

serde-rs/serde#204 (comment)

Honor docker ignore file

Implement import

See over lifetime parameter and borrowing API

Hi, sorry for the drive through comment; noticed this while answering questions from users of your library. I hope this helps.

In shiplift 0.3.2 I see this function:

pub fn get(&'a self, name: &'a str) -> Container

using 'a like this -- it's not a parameter of the current method, but of the whole impl block -- is almost never what you want. The result is that the code is asking for far more strict borrowing rules than are actually required.

Remember: If a lifetime parameter is used in two places, those two borrows must have the same lifetime.

• &'a self says that the borrow of self just to call this method must be the same length as the whole borrow of the reference that Containers is already storing.

  • What you actually want: The borrow of self can be shorter too, that's fine and applies in many more situations

• The &'a str says that the borrow of the string, too, must be of the same scope and lifetime. It adds another variable into the same borrow, which just makes it even more complicated to find a situation where they all live that same long time.

What you probably want:

    // I'm giving the lifetimes unconventional names here
    // so that we see them differently
    pub fn get<'thestring>(&self, name: &'thestring str) -> Container<'a, 'thestring> {
        Container::new(self.docker, name)
    }

What happened here? The &'a Docker is just a shared reference (im/mutability is important), so we just copy it along. Don't involve self in this borrow. Just read & copy the reference to the output.

The string gets its own lifetime variable. Now the signature describes the most general (and most versatile) borrowing behavior that is actually happening.

I went into the details for this method, and I hope it helps with fixing a few issues here and there in the API.

Hello, noob here. What happend with this error : thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Http(Io(Error { repr: Os { code: 2, message: "No such file or directory" } }))', /checkout/src/libcore/result.rs:860 note: Run with `RUST_BACKTRACE=1` for a backtrace. and what should i do to fix this error, thanks before

contribute tls back to hyper

Error to get stats -> MissingFieldError("swap")

I'm trying to run the following snippet of code:

extern crate shiplift;

use std::env;

fn main() {
    let docker = shiplift::Docker::new();
    let containers = docker.containers();
    let stats = containers.get("101d3870c70b").stats().expect("Won't work.");
    for s in stats {
            println!("{:?}", s);
    }
}

But rust is returning the following error:

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: MissingFieldError("swap")', /checkout/src/libcore/result.rs:916:5
note: Run with `RUST_BACKTRACE=1` for a backtrace.

How can I fix this?

BadRequest: "Bad parameter"

I'm getting a Fault { code: BadRequest, message: "bad parameter" } on every request to docker.

Here a two complete backtraces, created by running the examples from master:

[masch@mark-laptop:~/code/rust/shiplift on master]
% sudo RUST_LOG=debug RUST_BACKTRACE=1 ./target/debug/examples/events
listening for events
thread '<main>' panicked at 'called `Result::unwrap()` on an `Err` value: Fault { code: BadRequest, message: "bad parameter" }', ../src/libcore/result.rs:746
stack backtrace:
   1:     0x561838fb30b0 - sys::backtrace::tracing::imp::write::h3675b4f0ca767761Xcv
   2:     0x561838fb584b - panicking::default_handler::_$u7b$$u7b$closure$u7d$$u7d$::closure.44519
   3:     0x561838fb54b8 - panicking::default_handler::h18faf4fbd296d909lSz
   4:     0x561838faa47c - sys_common::unwind::begin_unwind_inner::hfb5d07d6e405c6bbg1t
   5:     0x561838faa908 - sys_common::unwind::begin_unwind_fmt::h8b491a76ae84af35m0t
   6:     0x561838fb2661 - rust_begin_unwind
   7:     0x561838fe4e5f - panicking::panic_fmt::h98b8cbb286f5298alcM
   8:     0x561838e7b610 - result::unwrap_failed::h14728117271176524093
                        at ../src/libcore/macros.rs:29
   9:     0x561838e7b3b4 - result::Result<T, E>::unwrap::h15254061339421577925
                        at ../src/libcore/result.rs:687
  10:     0x561838e7a51a - main::hf27ff4eeae946ce8gaa
                        at examples/events.rs:8
  11:     0x561838fb5114 - sys_common::unwind::try::try_fn::h14622312129452522850
  12:     0x561838fb25eb - __rust_try
  13:     0x561838fb4bab - rt::lang_start::h0ba42f7a8c46a626rKz
  14:     0x561838e7d879 - main
  15:     0x7f2f8563661f - __libc_start_main
  16:     0x561838e7a268 - _start
  17:                0x0 - <unknown>
[masch@mark-laptop:~/code/rust/shiplift on master]
% sudo RUST_LOG=debug RUST_BACKTRACE=1 ./target/debug/examples/containers
DEBUG:hyper::http::h1: request line: Get "/containers/json" Http11
DEBUG:hyper::http::h1: headers=Headers { Host: ///var/run/docker.sock:0, }
DEBUG:hyper::client::response: version=Http11, status=BadRequest
DEBUG:hyper::client::response: headers=Headers { Connection: close, Content-Type: text/plain, }
thread '<main>' panicked at 'called `Result::unwrap()` on an `Err` value: Fault { code: BadRequest, message: "bad parameter" }', ../src/libcore/result.rs:746
stack backtrace:
   1:     0x55e1a24fcab0 - sys::backtrace::tracing::imp::write::h3675b4f0ca767761Xcv
   2:     0x55e1a24ff24b - panicking::default_handler::_$u7b$$u7b$closure$u7d$$u7d$::closure.44519
   3:     0x55e1a24feeb8 - panicking::default_handler::h18faf4fbd296d909lSz
   4:     0x55e1a24f321c - sys_common::unwind::begin_unwind_inner::hfb5d07d6e405c6bbg1t
   5:     0x55e1a24f36a8 - sys_common::unwind::begin_unwind_fmt::h8b491a76ae84af35m0t
   6:     0x55e1a24fc061 - rust_begin_unwind
   7:     0x55e1a252e98f - panicking::panic_fmt::h98b8cbb286f5298alcM
   8:     0x55e1a22dad14 - result::unwrap_failed::h17605680571608167188
                        at ../src/libcore/macros.rs:29
   9:     0x55e1a22daab0 - result::Result<T, E>::unwrap::h8036330443532157769
                        at ../src/libcore/result.rs:687
  10:     0x55e1a22d729d - main::h94c991ab960693d3haa
                        at examples/containers.rs:9
  11:     0x55e1a24feb14 - sys_common::unwind::try::try_fn::h14622312129452522850
  12:     0x55e1a24fbfeb - __rust_try
  13:     0x55e1a24fe5ab - rt::lang_start::h0ba42f7a8c46a626rKz
  14:     0x55e1a22dcf09 - main
  15:     0x7f13eff5361f - __libc_start_main
  16:     0x55e1a22d70a8 - _start
  17:                0x0 - <unknown>

I tried to debug it but wasn't lucky.
I'm using docker 1.11.

Using curl works fine:

[masch@mark-laptop:~/code/rust/shiplift on master]
% sudo curl --unix-socket /var/run/docker.sock http:/containers/json -vvv 
*   Trying /var/run/docker.sock...
* Connected to http (/var/run/docker.sock) port 80 (#0)
> GET /containers/json HTTP/1.1
> Host: http
> User-Agent: curl/7.45.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Content-Type: application/json
< Server: Docker/1.11.0 (linux)
< Date: Thu, 21 Jul 2016 20:26:27 GMT
< Content-Length: 3
< 
[]
* Connection #0 to host http left intact

Any Ideas on what I'm doing wrong?

`LogsOptions` defaults are invalid

If you rely on the defaults for LogsOptions, the request returns a 400 Bad Request with the message "bad parameter".

I would suggest removing the #[derive(Default)] and implementing it manually to provide sane defaults. I haven't tested the other Options structs but I would imagine that one or two of them are afflicted with the same problem.

Bad documentation link in REDME.md !

It links to the old doc repository, the docs.rs one is more up to date !

where possible remove usage of `unwrap`

its an anti-pattern... and a trap

Expose container networks

    {
        "Id": "a0367a056c7f9d5c633ca3ef74a74b662105e103bf39efb3e71b69d38dd38a08",
        "Created": "2018-05-28T16:26:48.25834981Z",
        "Path": "/entrypoint.sh",
        "Args": [
            "/usr/bin/supervisord"
        ],
        ...
        "NetworkSettings": {
            ...
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {
                "foo": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": [
                        "app",
                        "a0367a056c7f"
                    ],
                    "NetworkID": "2dad4c8e55134d1a0fb9baeb2ab83de0500510e6a136be4c7248fb15c034f71d",
                    "EndpointID": "b3b1e155d80406597014df448b8c535fcfaf5d7df59899ecaf7f26e7e3548686",
                    "Gateway": "172.20.0.1",
                    "IPAddress": "172.20.0.4",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:ac:14:00:04",
                    "DriverOpts": null
                }
            }
        }
    }

It would be nice if the data under NetworkSettings > Networks was exposed to the API. As you can see above, my container doesn't have an IP on the host, but it does have an IP on the network it generally communicates over.

upgrade to hyper 0.9

Look into srcing info from api schema

https://github.com/docker/docker/blob/master/api/swagger.yaml

Unfriendly API?

Consider the following function:

fn make_container<'b>(containers: &'b Containers) -> Container<'b, 'b> {
    let cont_opts = ContainerOptions::builder("alpine")
        .cmd(vec!["true"])
        .build();
    let cont_info = containers.create(&cont_opts).unwrap();
    containers.get(&cont_info.Id)
}

It doesn't compile because the returned Container needs to have a reference to its Id. I'm not sure if it's even possible to do this.

Having more structs that own their data might make the API a bit easier to use.

add a pull request template

Why? To shorten the feedback for what's typically expected. To improve responsiveness. To remind contributors that small pulls are easier to review than big pulls.

Filter on multiples events of the same type

It's an awesome feature if we can listen on multiples events of the same type.

Like describe in docker-cli events command.

Using the same filter multiple times will be handled as a OR;
for example --filter container=588a23dac085 --filter container=a8f7720b8c22
will display events for container 588a23dac085 OR container a8f7720b8c22

In this example, I want all container events that start, stop or die. But I only get die event.
Look documentation

use shiplift::builder::EventsOptions;
use shiplift::builder::EventFilterType;
use shiplift::builder::EventFilter;

let mut options = EventsOptions::builder();
options.filter(vec![
    EventFilter::Type(EventFilterType::Container),
    EventFilter::Event("start".to_string()),
]);
options.filter(vec![
    EventFilter::Event("stop".to_string()),
]);
options.filter(vec![
   EventFilter::Event("die".to_string()),
]);

This is due to internal implementation that only send one filters parameter to docker daemon.

Tests...

All testing is currently done at the integration level. It would be useful to have some unit tests for things like serialization

Recreate container

I'm trying to write a tool that pulls an image and recreates an existing container.

To create a container, I need to provide a ContainerOptions instance.

Is there a way to extract the ContainerOptions from an existing container by inspecting it? Could we maybe impl From<ContainerDetails> for ContainerOptions?

Add support for 'docker commit'

Feature request for the docker commit command. I could really use it in my project.

name it

not married to shiplift. boxer maybe?

exec?

Would you be open to implementing (or accepting a PR to do so) the exec endpoint? Our use-case requires it.

Factor out tar into its own crate

Image delete rep

Implement export

Use error chain for deriving errors

Error chain is a very common errors handling crate that has a good ecosystem going for it. This library could benefit from being a friendly actor in that ecosystem

Make the API asynchronous

Since we're using hyper under the hood, which is fully async, it is a bit of a shame that shiplift's API isn't. It would be great to change the methods that make a call to Docker to return a Future of some kind.

Some examples of Rust crates with an Async API:

• fantoccini
• tokio-zookeeper

Maybe for version 0.5? :)

check style with cargo fmt in CI

It makes contributing easier and consistent if there is a consistent code style. Rustfmt is the tool to do that and there are ways of connecting this to CI. Let's do that

create a CONTRIBUTING.md file

Why? To further shorten the feedback loop for contributors for what's expected and for tighter integration with tooling/UX github provides. Some examples like wasm-pack provide good examples.

provide explicit arg constructors

Is there any examples on how to get all containers (including stopped/exited)?

Hello Folks,

Is there any example on how to get all containers of a host, including the stopped/exited?

For me, just the running containers are return by docker.containers().list

Add support for serializing hashes in ContainerOptions

Currently ContainerOptions#serialize can only deal with things that are key, value on the body of the message or key, value on HostConfig where each HostConfig key maps to an array of the values.

I think this falls down a bit when considering adding support for something like Devices where the value should be an array of key value pairs.

The remote API (v 1.24) specifies devices like this:

Devices - A list of devices to add to the container specified as a JSON object in the form

{ "PathOnHost": "/dev/deviceName", "PathInContainer": "/dev/deviceName", "CgroupPermissions": "mrw"}

Convert try! To ?

Add supported API version to README

I don't think that v1.26 of the remote API is currently supported, I think the README should be updated with information related to the minimum and max supported versions.

[Bug] Image::inspect() fails when there is no label on image.

When the dockerd daemon returns a json with:

{
   ...
   "Config":{
      ...
      "Labels":null
   },
   ...
}

This code returns a Err(Decoding(ExpectedError("Object", "null"))) error because the Json::decode() is trying to cast null into HashMap<String, String>:

shiplift/src/lib.rs

Lines 98 to 101 in 0618535

	pub fn inspect(&self) -> Result<ImageDetails> {
	let raw = self.docker.get(&format!("/images/{}/json", self.name)[..])?;
	Ok(json::decode::<ImageDetails>(&raw)?)
	}

This is why we should change this line

shiplift/src/rep.rs

Line 149 in 0618535

pub Labels: HashMap<String, String>,

to:

  pub Labels: Option<HashMap<String, String>>,

to permit null in the Labels field in the Json response.

Representing updates to Docker API

First, thanks for the library - using it has definitely increased our efficiency.

The specific use-case I would like to solve is to get access to a piece of container metadata that's returned with more recent Docker daemons: (--cpus): https://docs.docker.com/config/containers/resource_constraints/#configure-the-default-cfs-scheduler

I would be happy to provide a PR, but am not sure how to represent return values from the Docker daemon, given that different versions will return different values. Any thoughts on how such a thing might get accomplished?

Thanks!