vbpf / ebpf-verifier Goto Github PK

Having a licence (and copyright notices, if relevant) attached to the repository would let people know if, and under what conditions, they could reuse and/or modify the code from this project.

• Memory access
• Function arguments
• Dataflow

• add programs like simple_lt abd simple_neq which the current verifier succeeds on.

• add a simple_loop like program where we only use pointers (no integers. hopefully, go forward not backwards)

• program like mapcpy which reads length from start of packet, reads the rest of data of length bytes compute checksum and compare to other field

• (low priority) in mapcpy check if modulo access w/o loop works
  value1[i % VALUE_SIZE + 1] = value2[i % VALUE_SIZE];

• memset() resets from begining to end

• Cilium's examples
• OVS
• Linux samples
• p4-to-eBPF examples

Currently the verifier silently allow unreachable statements - most importantly, exit; but such cases are most likely a bug in the implementation.

The linux verifier disallow unreachable instructions. This seems to be overly restrictive. Moreover, we can't easily mimic that since the nondeterministic checking of regions on each load is intended to have some unreachable parts.

See https://elixir.bootlin.com/linux/v4.20/source/kernel/bpf/verifier.c#L2273

Maybe https://github.com/solarflarecom/ebpf_asm

Should help in #31

Communicate to crab the precise signedness of checks in e.g. branching.

Types may be incorporated into the syntax

Should be done before translation to crab, and independent from it.
This is unrelated to #37 , although both will make the translation to crab more straightforward.

There are a total of 4 examples in the ebpf-verifier-flops directory now. We may want to try them soonish and give William some feedback.

To keep code provenance clean, I would recommend an approach followed by many other open source projects used in production, which list the Copyright and License at the top of each file. It could be as short as two lines.

E.g., the Open Enclave SDK (for which I am a maintainer) uses this:

// Copyright (c) Open Enclave SDK contributors.
// Licensed under the MIT License.

The first line means that each contributor retains the copyright, and it intentionally does not have copyright any individual or organization since there may be pull requests from different people over time that touch the same file.

Kubernetes uses a similar 1-line copyright (The Kubernetes Authors.) but copies the Apache license into each file, as in this example:
https://github.com/kubernetes/kubernetes/blob/master/plugin/pkg/admission/gc/gc_admission.go

Iotivity (for which I was a maintainer) is similar, with a 1-line copyright by the original contributor of the file and copies the Apache license into each file, as in this example:
https://github.com/iotivity/iotivity/blob/master/resource/src/OCResource.cpp
The copyright statement being only the original contributor of each file is something many people don't like and why other projects usually don't follow that approach.

AllJoyn (for which I was a maintainer) similarly used a 1-line copyright by contributors and copied the Apache license into each file, as in this example:
https://github.com/alljoyn/core-alljoyn/blob/master/alljoyn_core/src/AllJoynCrypto.h

My opinion is that since the LICENSE file is in the root of the ebpf-verifier repo, and it's a standard OSI-approved license (and indeed it's the same license that the Open Enclave SDK uses), that the MIT license need not be copied into each file and a simple two-line comment would suffice.

I'd suggest something like this:
// Copyright (c) Prevail Verifier contributors.
// Licensed under the MIT License.

This would make it clear which files are MIT licensed vs which are not, and what the copyright is on each file.

The Open Enclave SDK also uses a git checker to require that every new file has such a notice at the top. I'd be happy to contribute that here if others agree with such an approach.

Useful for the dot output.
(not for the analysis since crab already has this).

Currently:

• 32 bit ops wrap around, but crab does not support that, so it's an havoc
• 64 bit ops do not wrap around. Havocing them is much too imprecise. Checking for UINT64_MAX is impossible since crab looks for an int, not a long long

Should we make the asm files a different project? an archive within this project? src/asm?

Currently the size of maps is assumed to be constant.

Since it's both load and store, it makes sense to decompose it to 3 instructions: load, add, store.

Potential queries:

• Enumerate access pattern
• Stats about access pattern
• Stats about general access pattern

Hello,

Initially, I could not reproduce Figures 9/10 due to scripts/makeplot.py throwing a ValueError.

Here is an output:

Traceback (most recent call last):
  File "scripts/makeplot.py", line 13, in <module>
    data = np.genfromtxt(sys.argv[1], delimiter=',', names=True)
  File "/home/rahatly/.local/lib/python3.6/site-packages/numpy/lib/npyio.py", line 2089, in genfromtxt
    raise ValueError(errmsg)
ValueError: Some errors were detected !
    Line #2 (got 26 columns instead of 45)
    Line #3 (got 26 columns instead of 45)
    Line #4 (got 26 columns instead of 45)
    Line #5 (got 26 columns instead of 45)
    ...

I removed last 19 columns which seemed to have empty rows. This fixed the problem.

Nevertheless, I still cannot reproduce figures 9 & 10. I attached the generated plots below. Would you suggest a fix or any other script I could take a look at?

Thanks!

Specifically: https://elixir.bootlin.com/linux/v4.18-rc6/source/kernel/bpf/helpers.c#L179

The installation steps described in the README fail with Ubuntu 18.04 because the version of libboost-dev in the Ubuntu repositories is too old:

src/main_check.cpp:6:10: fatal error: boost/container_hash/hash.hpp: No such file or directory
 #include <boost/container_hash/hash.hpp>
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

boost/container_hash/hash.hpp was introduced in version 1.67.0 of libboost, the Ubuntu repositories have version 1.65:

$ apt-cache policy libboost-dev
libboost-dev:
  Installed: 1.65.1.0ubuntu1
  Candidate: 1.65.1.0ubuntu1
  Version table:
 *** 1.65.1.0ubuntu1 500
        500 http://fr.archive.ubuntu.com/ubuntu bionic/main amd64 Packages
        100 /var/lib/dpkg/status

I managed to build the project after installing libboost 1.70.0 from sources as follows:

wget -q https://dl.bintray.com/boostorg/release/1.70.0/source/boost_1_70_0.tar.bz2
tar xvfo boost_1_70_0.tar.bz2
cd boost_1_70_0
./bootstrap.sh --with-libraries=atomic,chrono,context,coroutine,date_time,exception,filesystem,graph,graph_parallel,iostreams,locale,log,math,mpi,program_options,random,regex,serialization,system,test,thread,timer,wave
./b2 toolset=gcc cxxflags=-std=gnu++0x -j4
./b2 install

Do you prefer me to update the README with the installation-from-sources steps or using an alternative repository? Or maybe just specify the version of Ubuntu needed (18.10?)?

My understanding is that the src/crab directory is a Derivative Work based on the https://github.com/seahorn/crab library (and this commit among others does confirm that).
The https://github.com/seahorn/crab code LICENSE is Apache 2.0 where clause 4 says:

4. Redistribution. You may reproduce and distribute copies of the
   Work or Derivative Works thereof in any medium, with or without
   modifications, and in Source or Object form, provided that You
   meet the following conditions:

   (a) You must give any other recipients of the Work or
   Derivative Works a copy of this License; and

   (b) You must cause any modified files to carry prominent notices
   stating that You changed the files; and

   (c) You must retain, in the Source form of any Derivative Works
   that You distribute, all copyright, patent, trademark, and
   attribution notices from the Source form of the Work,
   excluding those notices that do not pertain to any part of
   the Derivative Works; and

   (d) If the Work includes a "NOTICE" text file as part of its
   distribution, then any Derivative Works that You distribute must
   include a readable copy of the attribution notices contained
   within such NOTICE file, excluding those notices that do not
   pertain to any part of the Derivative Works, in at least one
   of the following places: within a NOTICE text file distributed
   as part of the Derivative Works; within the Source form or
   documentation, if provided along with the Derivative Works; or,
   within a display generated by the Derivative Works, if and
   wherever such third-party notices normally appear. The contents
   of the NOTICE file are for informational purposes only and
   do not modify the License. You may add Your own attribution
   notices within Derivative Works that You distribute, alongside
   or as an addendum to the NOTICE text from the Work, provided
   that such additional attribution notices cannot be construed
   as modifying the License.

   You may add Your own copyright statement to Your modifications and
   may provide additional or different license terms and conditions
   for use, reproduction, or distribution of Your modifications, or
   for any such Derivative Works as a whole, provided Your use,
   reproduction, and distribution of the Work otherwise complies with
   the conditions stated in this License.

I didn't see anything for which parts (c) or (d) would apply but requirements (a) and (b) do apply.
(a) applies since the ebpf-verifier is otherwise under the MIT license rather than Apache (which is permitted by the last paragraph only if the Apache license is also provided for this library).

For (b), some project liberally interpret that such that a github history is considered prominent notice. crab_utils\heap.hpp is an example of a file that does contain a prominent notice in it, but other crab files do not.

Furthermore, some parts of the Crab library in the ebpf-verifier project carry a notice like this one from src/crab/wto.hpp:

• Author: Arnaud J. Venet ([email protected])
• Contributors: Jorge A. Navas ([email protected])
• Notices:
• Copyright (c) 2011 United States Government as represented by the
• Administrator of the National Aeronautics and Space Administration.
• All Rights Reserved.

Does "All Rights Reserved" preclude the ability to create derivative works
as the ebpf-verifier did? Presumably Jorge as a listed contributor acquired
such right, and this should probably be stated explicitly somewhere.

Jorge should probably weigh in with his preferred resolution to these crab issues.

Affects e.g. reject_helper_access_to_packet_test7_cls_unchecked_packetptr.3.

This check should be very similar to access on store/load

It will require some interface to define which to analyse.

Hi,

Very cool work! On my Ubuntu 18.04 system if I compile with g++ (7 and 8) versions it complains

build/asm_marshal.o <- src/asm_marshal.cpp
src/asm_marshal.cpp: In member function ‘std::vector<ebpf_inst> MarshalVisitor::makeLddw(Reg, bool, int32_t, int32_t)’:
src/asm_marshal.cpp:78:40: sorry, unimplemented: non-trivial designated initializers not supported
             ebpf_inst{ .imm = next_imm }
                                        ^
compilation terminated due to -Wfatal-errors.
Makefile:96: recipe for target 'build/asm_marshal.o' failed
make: *** [build/asm_marshal.o] Error 1

https://github.com/doslab/gcc-designated-initializer-support-cpp/wiki/GCC-Support-for-Non-trivial-designated-initializer-in-CPP

However, if I compile with make CXX=clang++, it works (as far as I can tell, need to do more testing).

Implications can be encoded by adding a "detour" with the assumption and the assertion.