opencontainers / runtime-tools Goto Github PK

After the TOB finishes voting, we will do a repo rename here to "runtime-tools"
https://groups.google.com/a/opencontainers.org/forum/#!forum/tob

This is based on the new OCI Tooling project proposal:
https://github.com/opencontainers/tob/blob/master/proposals/tools.md

Just wanted to know, if there is any specific reason for cli instead of cobra as we can reduce some of the code in generate.go using cobra. Something like

if context.IsSet("read-only") {
g.SetRootReadonly(context.Bool("read-only"))
}

could be set as :
cmd.Flags().BoolVar(&context.readonly, "read-only", context.readonly "make the container's rootfs read-only.")
where cmd here points to oci-runtime-tool generate

I may be missing an obvious point here as I am new to code base. Please, feel free to correct me in case I made some mistake.

While github.com/opencontainers/runtime-spec/specs-go is a reasonable name for the package, the Spec type there should really have been called Config (because it represents a particular config.json). It's probably too late to be worth changing in runtime-spec, but I think it is worth updating ocitools to use config in its internal names and public API while those are still fluxy. Thoughts?

oci-runtime-tools has semantic versions already and a CI process. It should be extended to include publishing release binaries to github.

Currently we're using Go, which means it's hard to build a single validator that can process multiple versions of the spec, because you'd have to use tedious interface{} manipulation or load separate packages with structures for each supported version.

We're also not using a test suite, which means it's hard to test multiple aspects of the configuration and report on several errors (the current code just dies on the first error).

By using Python's unittest, we get more convenient handling of generic JSON and an established test framework that doesn't need a lot of boilerplate. I've started working up this approach here if folks want to kick the tires. Examples of potentially tricky things and how this approach lets us handle them easily:

• Skipping particular tests if the configuration has an unrecognized version (wking/oci-runtime-config-validator@c607380f).
• Applying the Windows-specific mount-nesting restriction (here).
• Applying the 0.5.0-specific (and previous, but I haven't bothered supporting them) relative root.path requirement (here).

Example compact output with the current tip (wking/oci-runtime-config-validator@cd0facf0a. I still haven't finished process, later entries in config.md or anything from the platform-specific files):

$ BUNDLE=~/src/opencontainers/my-app python3 -m unittest
...s.............
----------------------------------------------------------------------
Ran 17 tests in 0.006s

OK (skipped=1)

and verbose output:

$ BUNDLE=~/src/opencontainers/my-app python3 -m unittest -v
test_configuration (test.test_bundle.TestBundle)
config.json MUST reside in the root of the bundle directory. ... ok
test_root (test.test_bundle.TestBundle)
The bundle directory MUST contain the root filesystem. ... ok
test_destination (test.test_mounts.TestMounts)
destination (string, required). ... ok
test_destination_nesting (test.test_mounts.TestMounts)
Mount destinations MUST not be nested within another mount. ... skipped 'the destination-nesting restriction only applies to Windows for specification version 1.0.0-rc1.'
test_options (test.test_mounts.TestMounts)
options (list of strings, optional). ... ok
test_source (test.test_mounts.TestMounts)
source (string, required). ... ok
test_type (test.test_mounts.TestMounts)
type (string, required). ... ok
test_args (test.test_process.TestProcess)
args (array of strings, required). ... ok
test_cwd (test.test_process.TestProcess)
cwd (string, required). ... ok
test_env (test.test_process.TestProcess)
env (array of strings, optional). ... ok
test_process (test.test_process.TestProcess)
process (object, required). ... ok
test_terminal (test.test_process.TestProcess)
terminal (bool, optional). ... ok
test_path (test.test_root.TestRoot)
path (string, required). ... ok
test_readonly (test.test_root.TestRoot)
readonly (bool, optional). ... ok
test_syntax (test.test_syntax.TestSyntax)
All configuration JSON MUST be encoded in UTF-8. ... ok
test_recognized_version (test.test_version.TestVersion)
Check for a recognized configuration version. ... ok
test_semantic_version (test.test_version.TestVersion)
ociVersion (string, required) MUST be in SemVer v2.0.0 format. ... ok

----------------------------------------------------------------------
Ran 17 tests in 0.007s

OK (skipped=1)

And with an unrecognized version:

$ BUNDLE=~/src/opencontainers/my-app python3 -m unittest
.sssssssssssss.F.
======================================================================
FAIL: test_recognized_version (test.test_version.TestVersion)
Check for a recognized configuration version.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/wking/src/opencontainers/oci-runtime-config-validator/test/test_version.py", line 39, in test_recognized_version
    .format(util.VERSION))
AssertionError: '1.0.0-rc2' not found in ['1.0.0-rc1', '0.5.0'] : Unrecognized configuration version.  Either your configuration does not match an OCI specification or the test suite has not been taught to process the version you are using.

----------------------------------------------------------------------
Ran 17 tests in 0.002s

FAILED (failures=1, skipped=13)
$ BUNDLE=~/src/opencontainers/my-app python3 -m unittest -v
test_configuration (test.test_bundle.TestBundle)
config.json MUST reside in the root of the bundle directory. ... ok
test_root (test.test_bundle.TestBundle)
The bundle directory MUST contain the root filesystem. ... skipped 'cannot validate an unrecognized version'
test_destination (test.test_mounts.TestMounts)
destination (string, required). ... skipped 'cannot validate an unrecognized version'
test_destination_nesting (test.test_mounts.TestMounts)
Mount destinations MUST not be nested within another mount. ... skipped 'cannot validate an unrecognized version'
test_options (test.test_mounts.TestMounts)
options (list of strings, optional). ... skipped 'cannot validate an unrecognized version'
test_source (test.test_mounts.TestMounts)
source (string, required). ... skipped 'cannot validate an unrecognized version'
test_type (test.test_mounts.TestMounts)
type (string, required). ... skipped 'cannot validate an unrecognized version'
test_args (test.test_process.TestProcess)
args (array of strings, required). ... skipped 'cannot validate an unrecognized version'
test_cwd (test.test_process.TestProcess)
cwd (string, required). ... skipped 'cannot validate an unrecognized version'
test_env (test.test_process.TestProcess)
env (array of strings, optional). ... skipped 'cannot validate an unrecognized version'
test_process (test.test_process.TestProcess)
process (object, required). ... skipped 'cannot validate an unrecognized version'
test_terminal (test.test_process.TestProcess)
terminal (bool, optional). ... skipped 'cannot validate an unrecognized version'
test_path (test.test_root.TestRoot)
path (string, required). ... skipped 'cannot validate an unrecognized version'
test_readonly (test.test_root.TestRoot)
readonly (bool, optional). ... skipped 'cannot validate an unrecognized version'
test_syntax (test.test_syntax.TestSyntax)
All configuration JSON MUST be encoded in UTF-8. ... ok
test_recognized_version (test.test_version.TestVersion)
Check for a recognized configuration version. ... FAIL
test_semantic_version (test.test_version.TestVersion)
ociVersion (string, required) MUST be in SemVer v2.0.0 format. ... ok

======================================================================
FAIL: test_recognized_version (test.test_version.TestVersion)
Check for a recognized configuration version.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/wking/src/opencontainers/oci-runtime-config-validator/test/test_version.py", line 39, in test_recognized_version
    .format(util.VERSION))
AssertionError: '1.0.0-rc2' not found in ['1.0.0-rc1', '0.5.0'] : Unrecognized configuration version.  Either your configuration does not match an OCI specification or the test suite has not been taught to process the version you are using.

----------------------------------------------------------------------
Ran 17 tests in 0.003s

FAILED (failures=1, skipped=13)

This approach would also likely work fine in other languages and test frameworks that make it easy to handle generic JSON. I'm familiar with Python, but if the OCI community prefers a different language, I'm game to try. So:

1. Does this sound like a useful direction? If so,
2. Is it worth pulling into ocitools? Go is a reasonable language for compiled binaries like runtimetest (certainly a better choice for that than Python), and multi-language repositories can get awkward. The validation tasks (configuration validation and runtime validation) seem to decouple well, so I'd rather not stuff both into a single repository. But I can live with a single repo if it's the maintainer preference ;).

I have add the initial cod of runtimetest command, to replace runtime_test.sh.
Reference issuses:
Support test cases #7
Convert test_runtime.sh into a go program #8

Now, the result shows as below:

# ./ocitools runtimetest -r runc -o all
INFO[0001] successful Details:                          
INFO[0001] =============================================================================================== 
INFO[0001] 
BundleName:
  process1
BundleDir:
  bundles/process1
CaseArgs:
  --args=./runtimetest --rootfs=rootfs --read-only=true
TestResult:
  SUCCESS

INFO[0001] 
BundleName:
  hostname0
BundleDir:
  bundles/hostname0
CaseArgs:
  --args=./runtimetest --rootfs=rootfs --hostname=zenlin
TestResult:
  SUCCESS

INFO[0001] 
BundleName:
  process0
BundleDir:
  bundles/process0
CaseArgs:
  --args=./runtimetest --rootfs=rootfs --read-only=false
TestResult:
  SUCCESS

INFO[0001] failure Details:                             
INFO[0001] =============================================================================================== 
INFO[0001] =============================================================================================== 
INFO[0001] statistics:  3 bundles success, 0 bundles failed

My plan as below,

1. Reach cases in cases.conf
2. Support different runtime
   what is your suggestion about this? @mrunalp

oci-runtime-tool validate is unable to detect incorrect keywords in config.json before it runs. For example,

"linux": {
"rootPropagation": "shared"
...

instead of "rootfsPropagation".

Is there a newer version cuz when I use criu i got an error with the seccomp

If ocitools want to support different runtimes, there are two steps are needed.

1. wrappers to CLI, thus, ocitools runtimetest can call the runtime with the uniform API.
2. convert tools, similar to oci2aci.

we can disscuss the 1st in issuse "Initial code of runtimetest command #10".
And this topic talks about where to import convert tools, one way is to import convert tool as a subCommand of ocitools, like generate.

Anyway, ocitools is like a oci tool set now.

This is somewhat related to #16 (where a comment sounded like “we'll get serious once we get to 1.0”), and #69 (where I argue that we should not float with runC). But concretely, I am unable to test ccon with the current ocitools. For one, my command-line API is different (I expect RUNTIME start --id $CONTAINER_ID, while runC seems to prefer RUNTIME start $CONTAINER_ID). But once I've patched around that:

ocitools$ git diff -U0
diff --git a/test_runtime.sh b/test_runtime.sh
index da3f026..d498ae8 100755
--- a/test_runtime.sh
+++ b/test_runtime.sh
@@ -84 +84 @@ popd > /dev/null
-TESTCMD="${RUNTIME} start $(uuidgen)"
+TESTCMD="${RUNTIME} start --id $(uuidgen)"

I still cannot run the tests, because they're based on an unreleased 0.6.0-dev version of the spec:

ocitools$ ./test_runtime.sh -l debug -r ccon-oci
-----------------------------------------------------------------------------------
VALIDATING RUNTIME: ccon-oci
-----------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/wking/bin/ccon-oci", line 578, in <module>
    main()
  File "/home/wking/bin/ccon-oci", line 574, in main
    args.func(**kwargs)
  File "/home/wking/bin/ccon-oci", line 442, in start
    config_state = _load_config(bundle=bundle, id=id)
  File "/home/wking/bin/ccon-oci", line 421, in _load_config
    version = _get_version(config=config)
  File "/home/wking/bin/ccon-oci", line 64, in _get_version
    'unsupported OCI-spec version: {}'.format(version))
NotImplementedError: unsupported OCI-spec version: 0.6.0-dev
Runtime ccon-oci failed validation

Is there a branching plan for supporting per-version testing? For example, we might want to start a v1.0 branch to track the v1.0 release series while master continues on with any changes destined for v1.1 / v2.0, etc. Then the v1.0 branch could generate (at the moment) 1.0.0-rc1 specs that ccon-oci understands.

Spun off from #66, where “All configuration JSON MUST be encoded in UTF-8” is checked. And it seems to be:

$ ./ocitools generate
$ iconv -f UTF-8 -t UTF-16 config.json -o config-16.json
$ mv config-16.json config.json
$ file config.json
config.json: Little-endian UTF-16 Unicode text
$ ./ocitools validate --path .
FATA[0000] invalid character 'ÿ' looking for beginning of value

But I can't find Go docs on what input encodings are supported or UTF references in ocitools:

$ git grep -i utf origin/master
Binary file origin/master:rootfs.tar.gz matches

The Go docs reference RFC 4627, and the RFC explains how to detect the encoding, but maybe Go doesn't support that yet? Can someone with more Go+JSON experience explain how this works?

Currently, calling ocitools validate without specifying --path option would fail:

[hmeng@localhost test]$ ocitools validate 
FATA[0000] Bundle path shouldn't be empty  

I am thinking whether we should have a default value for the --path option of ocitools validate, the default value can be . (the current working dir).
This makes calling ocitools validate and ocitools generate consistent, also saves some typing.

For example:

17:33:40 grant in ~> ~ 
[*] oci-runtime-tool generate  --seccomp-default trace --seccomp-trace clone
default action already set as SCMP_ACT_TRACE

Instead, the default should be set to the specified action and a new rule just shouldn't be added specifically for the syscall... then the config should be printed as normal.

Other conflicting rules are just taken into account and generated gracefully. This doc explains the logic. That should probably be moved over to this repo as well.

No idea why it is not valid.

#390

The docker env-file parameter is very useful when running containers under systemd which require some amount of environment setup.

It would be extremely useful (given how runc is used) to be able to use the same parameter in runtime-tools when dynamically generating config.json bundles.

How to configure the cgroup memory swapiness using the ocitool or even directly in the config.json file.

Currently validation fails against v1.0.0-rc2 images, which is causing issues for adding oci-runtime-tools-based testing for umoci (cyphar/umoci#12).

The current error is just for kernelTCP, which was fixed in the new version of the spec. The only really annoying part of the update is that you need to handle the namespacing of all of the Go structs (where everything has Linux prepended to it now).

# oci-runtime-tool validate --path /tmp/bundle.a (status=1)
# time="2016-11-12T16:04:09Z" level=fatal msg="2 Errors detected:\n'Memory.KernelTCP' should not be empty.\ninternal error: validate currently only handles version 1.0.0-rc1-dev, but the supplied configuration targets 1.0.0-rc2"

I have seen the following inconsistencies between "ocitools generate" and "runc spec". So far:

terminal

runc spec defaults terminal to true, but ocitools defaults to false. I also think it should allow setting the terminal attribute.

rootfs readonly

ocitools generate defaults to true wheras runc spec defauilts to false

I will update this list, but for now here is an example generated by each tool: https://gist.github.com/amitsaha/3fed080c8f2de6f52082ce84e82656de

As I pointed out after #212 landed, we forgot to bump MAINTAINERS and nominate @liangchenye to put him back in.
#237 also suggests a maintainer change, but I think that should happen after this.

platform is removed from spec, so all the validation code should be removed.

The logic of ocitools --host-specific generate should be rechecked and behave as it is supposed to be.
Here are two examples:

1. if the user runs ocitools --host-specific generate --linux-cpu-shares=512 on a windows machine, the --linux-cpu-shares should be ignored.
2. if the user run ocitools --host-specific generate --output config.json on a windows machine, the config.json file should not include the linux section.

current master isn't working with runtime-spec@rc1 nor rc2 - generate above all should be updated

I think we can have a MAINTAINERS list on test/tooling project just like runc/specs.
I mentioned this on the TOB mailing list but seems no feedback from TOB members.
So maybe we discuss this just inside this project? @mrunalp

If you set an environment variable to a different value, you will get two entries in process.env. Apart from being an eyesore, POSIX doesn't actually specify what should happen if you have two variables set with the same name.

% oci-runtime-tool generate --env A=B --env A=C
...
                "env": [
                        "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                        "TERM=xterm",
                        "A=B",
                        "A=C"
                ],
...

IMO we should change the environment setting signature to something like AddProcessEnv(key, value string). Or we add parsing to the code so we replace entries if we see them.

It is the successor of #324.
We can wrap all the 'SHOULD' errors into one type(similar implementation of 'os.PathError') and then checks the returned type of errors with the log-level. If log-level is 'warning', we shoud also return the 'SHOULD testing' result.

I think we're pretty clear that we want a subreaper to collect the container exit code when we're validating runC and other runtimes that use local Linux namespacing/cgroups to implement runtime-spec. However, that approach will not work for runtimes that use VMs, or on Windows, or anywhere that the host cannot see the container process (opencontainers/runtime-spec#459). Do we have a plan for validating those runtimes? Are we going to have a validation approach that ignores the container process exit code and looks for a magic token in the container's stdout? Are we going to adjust the in-flight command line API (opencontainers/runtime-spec#513) to allow for a long-running create? Are we going to land an event operation (opencontainers/runtime-spec#508)? With a spec 1.0 approaching, we probably want to at least have a strategy for testing these alternative runtimes, even if we don't have all the tooling implemented yet.

GitHub is redirecting codegansta/cli to urfave/cli:

$ curl -sI https://github.com/codegangsta/cli | grep Location
Location: https://github.com/urfave/cli

#66 currently checks the terminal box with a reference to this code setting terminal to false in the default template. Grepping through the current master doesn't turn up anything that looks like tests though:

$ git describe --always
0b5e2eb
$ git grep -i terminal | grep -v Godeps
generate.go:            spec.Process.Terminal = context.Bool("tty")
generate.go:                    Terminal: false,

I think the current spec wording is untestably vague, and there's a recent runtime-spec issue that agrees (opencontainers/runtime-spec#494). One possible outcome is dropping the property from the spec. But in the face of all this uncertainty, I expect we want to:

• Uncheck the terminal box in #66 (ping @liangchenye).
• Avoid testing the property until we have testable language for it in the spec, and understand if/why we are going to keep it in the spec.

Filesystem bundle

• code This REQUIRED file MUST reside in the root of the bundle directory.
• code config.json : This REQUIRED file, which MUST be named config.json.
• code While these artifacts MUST all be present in a single directory on the local filesystem.
• code This directory MUST be referenced from within the config.json file.

Runtime and Lifecycle

• 🌓 code The container's runtime environment MUST be created according to the configuration in config.json.
• code Any updates to config.json after container is running MUST not affect the container.
• WIP The user specified process MUST be executed in the container.
• The container MUST be destroyed by undoing the steps performed during create phase (step 2)
• WIP Query State: state
• WIP This operation MUST generate an error if it is not provided the ID of a container.
• WIP This operation MUST return the state of a container as specified in the State section. In particular, the state MUST be serialized as JSON.

Start 🔒 discuss

• WIP This operation MUST generate an error if it is not provided a path to the bundle and the container ID to associate with the container.
• WIP If the ID provided is not unique across all containers within the scope of the runtime, or is not valid in any other way, the implementation MUST generate an error.
• code Using the data in config.json, that are in the bundle's directory, this operation MUST create a new container.This includes creating the relevant namespaces, resource limits, etc and configuring the appropriate capabilities for the container.
• code A new process within the scope of the container MUST be created as specified by the config.json file otherwise an error MUST be generated.
• WIP Attempting to start an already running container MUST have no effect on the container and MUST generate an error.

Kill

• This operation MUST generate an error if it is not provided the container ID.
• Attempting to send a signal to a container that is not running MUST have no effect on the container and MUST generate an error.
• This operation MUST send the specified signal to the process in the container.
• When the process in the container is stopped, irrespective of it being as a result of a kill operation or any other reason, the status property of this container MUST be stopped.

Delete

• This operation MUST generate an error if it is not provided the container ID.
• Attempting to delete a container that does not exist MUST generate an error.
• Attempting to delete a container whose process is still running MUST generate an error.
• Deleting a container MUST delete the resources that were created during the create step.
• Note that resources associated with the container, but not created by this container, MUST NOT be deleted.
• Once a container is deleted its ID MAY be used by a subsequent container.

Configuration

Specification version

• code ociVersion (string, required) must be in SemVer v2.0.0 format.

Root

• code On Windows, for Windows Server Containers, this field is REQUIRED and MUST be specified as a volume GUID path. For Hyper-V Containers, this field MUST be omitted.
• code On all other platforms, this field is REQUIRED. The value SHOULD be the conventional rootfs.
• code path (string, required) A directory MUST exist at the relative path declared by the field.
• code readonly (bool, optional) If true then the root filesystem MUST be read-only inside the container.
• code readonly Defaults to false.
• [code] On Windows, this field MUST be omitted or false.

Mounts

• WIP The runtime MUST mount entries in the listed order.
• Destination of mount point: path inside container. For the Windows operating system, one mount destination MUST NOT be nested within another mount. (Ex: c:\foo and c:\foo\bar).
• code destination (required)
• code mounts type (string, required)
• code source (string, required)

Process

• code process cwd (string, required) is the working directory that will be set for the executable. This value MUST be an absolute path.

• code Runtimes MUST ignore consoleSize if terminal is false or unset.

• code process env (array of strings, optional) Elements in the array are specified as Strings in the form "KEY=value". The left hand side must consist solely of letters, digits, and underscores _ .

• code 'capabilities` Any value which cannot be mapped to a relevant kernel interface MUST cause an error.

• code If oomScoreAdj is set, the runtime MUST set oom_score_adj to the given value.

• code If oomScoreAdj is not set, the runtime MUST NOT change the value of oom_score_adj.

• code If rlimits contains duplicated entries with same type, the runtime MUST error out.

Platform-specific configuration

 NOT IN RC5
 windows (object, OPTIONAL) Windows-specific configuration. This MUST be set if the target platform of this spec is windows.

• code Platform-specific configuration os (string, required) $GOOS.
• code Platform-specific configuration arch (string, required) $GOARCH.
• The runtime MUST generate an error if it does not support the configured os.
• The runtime MUST generate an error if it does not support the configured arch.
• Hooks Hooks MUST be called in the listed order.
• code Hooks path is required for a hook, args and env are optional.
• code Hooks paths are absolute
• [go map] Annotations MUST be a key-value map
• [go map] Keys MUST be strings. Keys MUST NOT be an empty string.
• [go map] Values MUST be strings.
• code Keys using the org.opencontainers namespace are reserved and MUST NOT be used by subsequent specifications.
• code Implementations that are reading/processing this configuration file MUST NOT generate an error if they encounter an unknown annotation key.
• code Runtimes that are reading or processing this configuration file MUST NOT generate an error if they encounter an unknown property. Instead they MUST ignore unknown properties.
• code Runtimes that are reading or processing this configuration file MUST generate an error when invalid or unsupported values are encountered.

Linux-specific Container Configuration

Default Filesystems

• code The following filesystems SHOULD be made available in each container's filesystem:

Path    Type
/proc   procfs
/sys    sysfs
/dev/pts    devpts
/dev/shm    tmpfs

Namespaces

• code path (string, OPTIONAL) - an absolute path to namespace file in the runtime mount namespace.
• The runtime MUST place the container process in the namespace associated with that path. The runtime MUST generate an error if path is not associated with a namespace of type.
• If path is not specified, the runtime MUST create a new container namespace of type type.
• If a namespace type is not specified in the namespaces array, the container MUST inherit the runtime namespace of that type.
• code If a namespaces field contains duplicated namespaces with same type, the runtime MUST generate an error.
• code hostname (optional) on Linux , you can only set this if your bundle creates a new UTS namespace
• code The runtime SHOULD NOT modify the ownership of referenced filesystems to realize the mapping.

Device Devices

• code In addition to any devices configured with this setting, the runtime MUST also supply 'dev/nul' .....

• code lists devices that MUST be available in the container.

• code If a file already exists at path that does not match the requested device, the runtime MUST generate an error.

• code The same type, major and minor SHOULD NOT be used for multiple devices.

• code capabilities (array of strings, optional) Valid values are the strings for capabilities defined in the man page

• code rlimits (array of rlimits, optional) Valid values for the 'type' field are the resources defined in the man page.

• Control groups The runtime MUST apply entries in the listed order.

• Control groups You must specify at least one of weight or leafWeight in a given entry, and can specify both. Planed

• code JSON All configuration JSON MUST be encoded in UTF-8.

Hi Mrunal,
I test runc today and find it fails in the process validation.
The reason seems it adds an extra 'audit_read' capability which is not defined in the config.json.
It is not clear in the runtime spec that extra configs should taken as an error and we don't check the unexpected environment variables in the environment validation.
So maybe it does not necessary to popup an 'Unexpected capability' error? @mrunalp

Ma Shimiao ([email protected]) has done quite a bit of work so far!

https://github.com/opencontainers/runtime-tools/commits?author=Mashimiao

cc: @mrunalp

How to use the tool to configure the IP address, and exported ports or a runc container ?

See here, with language from #33. The language looks like it's drifting towards the in-flight opencontainers/runtime-spec#395. But however that PR goes down, the container process is gone by the time the post-stop hooks run, because that process existing is the stop event. I'd suggest echoing the spec:

The post-stop hooks are called after the container process is stopped.

although I'd be a bit happier if the spec used “exits” instead of “is stopped” (to avoid confusion with SIGSTOP).

Many config settings have smaller legal ranges than their Go type implies (e.g. swappiness from 0 to 100). We should check for this and error out if users supply invalid values.

It looks like @rhatdan has started on some documentation in a /man folder. It would be good to link to this from the ReadMe

We should have a discussion whether it makes sense to move ocitools into the runtime-spec repo. As a plus, it would give the tools more visibility and make it easier to keep in sync with the runtime-spec. It would also potentially simplify the project structure by merging it into one project (instead of having two)

FYI: the tooling for testing the image-spec currently lives in the same repo:
https://github.com/opencontainers/image-spec/tree/master/cmd/oci-image-tool

Hi,

I can't go get opencontainers/runtime-tools/generate.

go get github.com/opencontainers/runtime-tools/generate

# github.com/opencontainers/runtime-tools/generate
$GOPATH/src/github.com/opencontainers/runtime-tools/generate/generate.go:67: undefined: specs.Rlimit
$GOPATH/src/github.com/opencontainers/runtime-tools/generate/generate.go:115: undefined: specs.Resources
$GOPATH/src/github.com/opencontainers/runtime-tools/generate/generate.go:123: undefined: specs.Namespace
$GOPATH/src/github.com/opencontainers/runtime-tools/generate/generate.go:140: undefined: specs.Device
$GOPATH/src/github.com/opencontainers/runtime-tools/generate/generate.go:487: undefined: specs.Action
$GOPATH/src/github.com/opencontainers/runtime-tools/generate/generate.go:591: undefined: specs.Syscall
$GOPATH/src/github.com/opencontainers/runtime-tools/generate/generate.go:594: undefined: specs.Syscall
$GOPATH/src/github.com/opencontainers/runtime-tools/generate/generate.go:598: undefined: specs.Syscall
$GOPATH/src/github.com/opencontainers/runtime-tools/generate/generate.go:996: undefined: specs.Namespace
$GOPATH/src/github.com/opencontainers/runtime-tools/generate/generate.go:1067: undefined: specs.Arg
$GOPATH/src/github.com/opencontainers/runtime-tools/generate/generate.go:598: too many errors 

Is this error due to some Godeps/vendoring mismatch regarding runtime-spec ?

Validators (e.g. the Nu Html Checker) often report errors with references to the specification requirement that was violated. To get a good user experience and protect against checking unspecified behavior, I think ocitools checks should be setup to link to the relevant spec wording for any check they perform. For example, a bundle with non-UTF-8 JSON would fail with a reference to this. If this sounds like a reasonable policy, we should write it up in the README or a CONTRIBUTING.md and open issues/PRs for adjusting any existing checks.

It may be useful to add a how-to / getting started doc for people who may want to run the OCI Tools on a new, clean VM. (Such a process may also be useful for replicable certification results.)

Please comment and tag me back if it would be helpful for me to create a PR.

With help from @mrunalp, below are instructions that worked for me:

1. mkdir ~/gosrc (could be any directory)
2. export GOPATH=~/gosrc
3. sudo apt-get install gccgo-go
4. go get github.com/opencontainers/ocitools
5. cd ~/gosrc/src/github.com/opencontainers/ocitools
6. make
7. sudo apt-get install go-md2man
8. sudo make install

The tmpfs property follows docker by default, which mounts --tmpfs filesystems noexec, nosuid, nodev.

These are not particularly useful defaults in a lot of circumstances (i.e. providing /tmp to a container) because a huge amount of software expects to be able to execute things out of such directories.

It would be convenient to support additional syntax which allow being explicit about the mount options to tmpfs - i.e. --tmpfs /tmp:defaults or --tmpfs /tmp:- for no special options, --tmpfs /tmp:nosuid for adding options.

The generate code could be converted into a library that could be consumed in other projects.

Discussed in #354. @wking recommends we can using TAG native diagnostics to tell the different compliance levels of validating messages.

We now use urfave/cli to process the arguments of process. But there is a problem that urfave/cli do not provide the order of arguments.

The order is so important in some case. Let's think about such a case:

The user do not want any default rlimit config, and wants to use his rlimit config.

In this case, the user needs to remove all default rlimit config, and then adds a rlimit to the config. So the user uses this command:

oci-runtime-tool generate --rlimits-remove-all --rlimits-add RLIMIT_NOFILE:10:10

But the result is not accepable. This command will generate a config file without any rlimit config.

Why it happends?

Because the arguments processor does not provides the order of 'rlimits-remove-all' and 'rlimits-add', we dont know which is the first.
In this time, the oci-runtime-tool process 'rlimits-remove-all' after 'rlimits-add', so it firstly adds a rlimits RLIMITS_NOFILE, then remove all rlimits.

How to fix

I have not found any way to get the order of arguments, so I think replace with a new arguments processor is the only way to solve this problem.

I submit this issue because I dont know which arguments processor we should to use. I think we can discuss about them.

container "7df3579a-af8b-402f-b8d3-bc502521841d" does not exist
Runtime runc failed validation

I think we should create container first before start it.
https://github.com/opencontainers/runtime-tools/blob/master/test_runtime.sh#L82

The build status in README.md is pointing to probably old repo name. It should be something like

https://travis-ci.org/opencontainers/runtime-tools

The code that landed in #20 is still live. I think we want to use gocapability to get all of the host's capabilities. We want that behind --host-specific (in flight with #119), because “supported capabilities” is Linux/host specific. We also want to PR gocapability with a way to list all capabilities (supported or not) so we can perform host-agnostic validation from low-capability or non-Linux hosts.

If you use --template generate will still output the default template.

Below are the steps I took (with help from Stephe Walli) to run the OCI Runtime-Tools on a freshly created Ubuntu VM.

It would be great to have someone give this a quick try and comment with:
(a) Does this work for you? (Hopefully yes :) )
(b) Are there any ways this could be simplified?

Thanks--
--Rob

0. Create a new VM

1. Check to see if you have git, go, and curl installed:

• git: git --version
• curl: curl --version
• go: go --version

1.1. Install Git if you don't have it
// TODO

1.2. Install Curl

1.3. Install Go if you don't have it:
curl -O https://storage.googleapis.com/golang/go1.6.3.linux-amd64.tar.gz
sudo tar -C /usr/local -xzf go1.6.3.linux-amd64.tar.gz
// Add the following to .profile 
// set -o vi 
// export PATH=$PATH:/usr/local/go/bin
// export GOPATH=$HOME/work

2. Set-up Directories:
   mkdir work
   cd work
   mkdir src
   cd src
   mkdir github.com
   cd github.com
   mkdir RobDolinMS (or your GitHub alias)
   cd RobDolinMS
   cd ..
   mkdir opencontainers
   cd opencontainers

3. Clone the runc and runtime-tools repos:
   git clone https://github.com/openconainers/runc
   git clone https://github.com/opencontainers/runtime-tools

4. Build runc:
   sudo apt-get install make
   cd runc
   sudo apt-get build-essential
   sudo apt-get install pkg-config
   sudo apt-get install libseccomp-dev
   make
   // You've just built runc! Woo!
   sudo make install
   which runc
   runc --version

5. Get Runtime-Tools dependencies:
   cd runtime-tools
   go get github.com/russross/blackfriday
   go install github.com/russross/blackfriday
   go get github.com/cpuguy83/go-md2man
   go install github.com/cpuguy83/go-md2man
   sudo install -m 755 $GOPATH/bin/go-md2man /usr/bin

6. Build Runtime-Tools:
   sudo make install
   // You have now built the runtime tools! Woo!

7. Run the runtime tools against runc:
   sudo ./test_runtime.sh -r runc -l debug

we should setup milestones for runtime-tools.
I want to create one milestone for v1.0.0.
The question is we should set due date to when is suitable. @opencontainers/runtime-tools-maintainers