capeprivacy / tf-trusted Goto Github PK

I followed the instructions to build custom op and began the docker container and got the message

2019-06-06 17:46:22 INFO tf_trusted_enclave.cc : 86 : Server started on port 50051

In a separate command window, I ran the following command

model_run.py --model_file normal_train2.pb \
               --batch_size 10 \
               --input_file test_input.npy \
               --input_name "image" \
               --output_name "ArgMax"

But, then I ended up receiving the following error.

Traceback (most recent call last): File "/home/hrishikesh/anaconda3/bin/model_run.py", line 7, in <module> exec(compile(f.read(), __file__, 'exec')) File "/home/hrishikesh/tf-trusted/tf_trusted_custom_op/model_run.py", line 27, in <module> model_module = tf.load_op_library(shared_object) File "/home/hrishikesh/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/load_library.py", line 61, in load_op_library lib_handle = py_tf.TF_LoadLibrary(library_filename) tensorflow.python.framework.errors_impl.NotFoundError: /home/hrishikesh/tf-trusted/tf_trusted_custom_op/model_enclave_op.so: undefined symbol: _ZN10tensorflow12OpDefBuilderC1ESs

Any idea as to why its occurring?

Thank you for resolving the build error for building custom-op. It did work, but unfortunately, I get this while running TF-Trusted using Docker.

hrishikesh@hrishi:~/tf-trusted$ sudo docker run -it --rm \
>   -v bazel-cache:/root/.cache/bazel \
>   -v `pwd`:/opt/my-project \
>   -w /opt/my-project \
>   -p 50051:50051/tcp -p 50051:50051/udp \
>   gcr.io/asylo-framework/asylo:buildenv-v0.3.4 \
>   bazel run --config=enc-sim //tf_trusted \
>   --incompatible_disallow_filetype=false --incompatible_disallow_data_transition=false
Starting local Bazel server and connecting to it...
ERROR: error loading package '': Encountered error while reading extension file 'asylo/bazel/asylo_deps.bzl': no such package '@com_google_asylo//asylo/bazel': java.io.IOException: Error downloading [https://github.com/google/asylo/archive/v0.3.4.2.tar.gz] to /root/.cache/bazel/_bazel_root/b6bc0828d5b43c7085b650e0698bf388/external/com_google_asylo/v0.3.4.2.tar.gz: Checksum was 82226be212b9f3e2fb14fdf9223e4f376df89424874ac45faff215fa1027797e but wanted e408c614ad129dd7dff0dc7a816f77aae81f22eb851f63fc0bba7de61a467b62
ERROR: error loading package '': Encountered error while reading extension file 'asylo/bazel/asylo_deps.bzl': no such package '@com_google_asylo//asylo/bazel': java.io.IOException: Error downloading [https://github.com/google/asylo/archive/v0.3.4.2.tar.gz] to /root/.cache/bazel/_bazel_root/b6bc0828d5b43c7085b650e0698bf388/external/com_google_asylo/v0.3.4.2.tar.gz: Checksum was 82226be212b9f3e2fb14fdf9223e4f376df89424874ac45faff215fa1027797e but wanted e408c614ad129dd7dff0dc7a816f77aae81f22eb851f63fc0bba7de61a467b62
INFO: Elapsed time: 12.087s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
FAILED: Build did NOT complete successfully (0 packages loaded)
hrishikesh@hrishi:~/tf-trusted$ sudo docker run -it --rm   -v bazel-cache:/root/.cache/bazel   -v `pwd`:/opt/my-project   -w /opt/my-project   -p 50051:50051/tcp -p 50051:50051/udp   gcr.io/asylo-framework/asylo:buildenv-v0.3.4   bazel run --config=enc-sim //tf_trusted   --incompatible_disallow_filetype=false --incompatible_disallow_data_transition=false
Starting local Bazel server and connecting to it...
ERROR: error loading package '': Encountered error while reading extension file 'asylo/bazel/asylo_deps.bzl': no such package '@com_google_asylo//asylo/bazel': java.io.IOException: Error downloading [https://github.com/google/asylo/archive/v0.3.4.2.tar.gz] to /root/.cache/bazel/_bazel_root/b6bc0828d5b43c7085b650e0698bf388/external/com_google_asylo/v0.3.4.2.tar.gz: Checksum was 82226be212b9f3e2fb14fdf9223e4f376df89424874ac45faff215fa1027797e but wanted e408c614ad129dd7dff0dc7a816f77aae81f22eb851f63fc0bba7de61a467b62
ERROR: error loading package '': Encountered error while reading extension file 'asylo/bazel/asylo_deps.bzl': no such package '@com_google_asylo//asylo/bazel': java.io.IOException: Error downloading [https://github.com/google/asylo/archive/v0.3.4.2.tar.gz] to /root/.cache/bazel/_bazel_root/b6bc0828d5b43c7085b650e0698bf388/external/com_google_asylo/v0.3.4.2.tar.gz: Checksum was 82226be212b9f3e2fb14fdf9223e4f376df89424874ac45faff215fa1027797e but wanted e408c614ad129dd7dff0dc7a816f77aae81f22eb851f63fc0bba7de61a467b62
INFO: Elapsed time: 3.521s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
FAILED: Build did NOT complete successfully (0 packages loaded)

I followed the steps for installing tf_trusted_custom_op and it builds successfully but when I try to run model_run.py , I get the error ModuleNotFoundError: No module named 'tf_trusted_custom_op'

`root@4811bb1b5421:/opt/my-project/tf_trusted_custom_op# bazel build model_enclave_op.so
WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown".
WARNING: /root/.cache/bazel/_bazel_root/6a072cedc59c5d9384722d447b964014/external/local_config_tf/BUILD:3588:1: target 'libtensorflow_framework.so' is both a rule and a file; please choose another name for the rule
INFO: SHA256 (https://github.com/nanopb/nanopb/archive/f8ac463766281625ad710900479130c7fcb4d63b.tar.gz) = 8bbbb1e78d4ddb0a1919276924ab10d11b631df48b657d960e0c795a25515735
DEBUG: /root/.cache/bazel/_bazel_root/6a072cedc59c5d9384722d447b964014/external/bazel_tools/tools/build_defs/repo/http.bzl:43:9: ctx.attr.build_file @com_github_grpc_grpc//third_party:nanopb.BUILD, path /root/.cache/bazel/_bazel_root/6a072cedc59c5d9384722d447b964014/external/com_github_grpc_grpc/third_party/nanopb.BUILD
INFO: SHA256 (https://github.com/c-ares/c-ares/archive/3be1924221e1326df520f8498d704a5c4c8d0cce.tar.gz) = e69e33fd40a254fcf00d76efa76776d45f960e34307bd9cea9df93ef79a933f1
DEBUG: /root/.cache/bazel/_bazel_root/6a072cedc59c5d9384722d447b964014/external/bazel_tools/tools/build_defs/repo/http.bzl:43:9: ctx.attr.build_file @com_github_grpc_grpc//third_party:cares/cares.BUILD, path /root/.cache/bazel/_bazel_root/6a072cedc59c5d9384722d447b964014/external/com_github_grpc_grpc/third_party/cares/cares.BUILD
INFO: SHA256 (https://github.com/madler/zlib/archive/cacf7f1d4e3d44d871b605da3b647f07d718623f.tar.gz) = 6d4d6640ca3121620995ee255945161821218752b551a1a180f4215f7d124d45
DEBUG: /root/.cache/bazel/_bazel_root/6a072cedc59c5d9384722d447b964014/external/bazel_tools/tools/build_defs/repo/http.bzl:43:9: ctx.attr.build_file @com_github_grpc_grpc//third_party:zlib.BUILD, path /root/.cache/bazel/_bazel_root/6a072cedc59c5d9384722d447b964014/external/com_github_grpc_grpc/third_party/zlib.BUILD
INFO: SHA256 (https://boringssl.googlesource.com/boringssl/+archive/afc30d43eef92979b05776ec0963c9cede5fb80f.tar.gz) = d01d090d4a849f6b124651a2e48ea5766f3a155403ccad14f9fd92ffdd87d2d8
WARNING: /root/.cache/bazel/_bazel_root/6a072cedc59c5d9384722d447b964014/external/local_config_tf/BUILD:5:12: in hdrs attribute of cc_library rule @local_config_tf//:tf_header_lib: file '_api_implementation.so' from target '@local_config_tf//:tf_header_include' is not allowed in hdrs
WARNING: /root/.cache/bazel/_bazel_root/6a072cedc59c5d9384722d447b964014/external/local_config_tf/BUILD:5:12: in hdrs attribute of cc_library rule @local_config_tf//:tf_header_lib: file '_message.so' from target '@local_config_tf//:tf_header_include' is not allowed in hdrs
INFO: Analysed target //:model_enclave_op.so (21 packages loaded).
INFO: Found 1 target...
INFO: From SkylarkAction external/com_github_grpc_grpc/src/proto/grpc/reflection/v1alpha/reflection.pb.h:
bazel-out/k8-fastbuild/genfiles/external/com_github_grpc_grpc/external/com_github_grpc_grpc: warning: directory does not exist.
INFO: From SkylarkAction external/com_github_grpc_grpc/src/proto/grpc/reflection/v1alpha/reflection.grpc.pb.h:
bazel-out/k8-fastbuild/genfiles/external/com_github_grpc_grpc/external/com_github_grpc_grpc: warning: directory does not exist.
Target //:model_enclave_op.so up-to-date:
bazel-bin/model_enclave_op.so
INFO: Elapsed time: 65.391s, Critical Path: 8.76s
INFO: 1090 processes: 1090 local.

Hi,

I am trying to build the tf-trusted for custom op to run models inside SGX enclave. I am using an Ubuntu 14.04 machine.
The first script to build the custom operation worked, but the build fails when executing:
bazel build model_enclave_op.so

with this error:

INFO: From SkylarkAction external/com_github_grpc_grpc/src/proto/grpc/reflection/v1alpha/reflection.grpc.pb.h:
bazel-out/k8-fastbuild/genfiles/external/com_github_grpc_grpc/external/com_github_grpc_grpc: warning: directory does not exist.
ERROR: /opt/my-project/tf_trusted_custom_op/BUILD:17:1: C++ compilation of rule '//:model_enclave_op.so' failed (Exit 1): gcc failed: error executing command 
  (cd /root/.cache/bazel/_bazel_root/6a072cedc59c5d9384722d447b964014/execroot/tf_trusted_custom_op && \
  exec env - \
    PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
    PWD=/proc/self/cwd \
    TF_HEADER_DIR=/usr/local/lib/python2.7/dist-packages/tensorflow/include \
    TF_SHARED_LIBRARY_DIR=/usr/local/lib/python2.7/dist-packages/tensorflow \

Could you please advise? Thanks in advance!

Hi,

I am trying to test tf-trusted with some simple cases, but I don't know how to start. I find a logistic regression from tf-encrypted. I want to test more cases, like mnist with cnn. However, I don't know how to fill the arguments, e.g., input_name and output_name.

Any suggestion?

Asylo has upgraded the version of bazel used inside their docker container. This version of bazel is no longer compatible with the version of protobuf referenced by tensorflow 1.12.0. Tensorflow 1.13.1 is using the same version of protobuf as Asylo so once that is released should be able to upgrade to Asylo 0.3.4.

I've succeeded in installing all of the required parts but when I try to run a model (either in sim or on an enclave) I get

2020-07-23 15:14:19.438040: F bazel-out/k8-fastbuild/genfiles/external/local_config_tf/include/tensorflow/core/lib/core/refcount.h:90] Check failed: ref_.load() == 0 (1 vs. 0)

As far as I'm aware this tends to happen when adding new custom ops without the -DNDEBUG flag, as referenced here:

tensorflow/tensorflow#17316

Do you have an idea on how to fix this ? I'm unsure on where to add the flag

Thanks!

Dears,

I try the build ( refer https://github.com/dropoutlabs/tf-trusted#build-and-run-tf-trusted ) and failed with following error message.

Could you help with this ?

$ docker run -it --name tf-trusted --rm
-v droplab-bazel-cache:/root/.cache/bazel
-v pwd:/opt/my-project
-w /opt/my-project
-p 50051:50051/tcp -p 50051:50051/udp
gcr.io/asylo-framework/asylo
bazel run --config=enc-sim //tf_trusted
Starting local Bazel server and connecting to it...
INFO: Invocation ID: 3e917c6f-8fa6-4d54-aaac-78acb786e384
ERROR: /root/.cache/bazel/_bazel_root/b6bc0828d5b43c7085b650e0698bf388/external/com_google_protobuf/BUILD:599:1: Traceback (most recent call last):
File "/root/.cache/bazel/_bazel_root/b6bc0828d5b43c7085b650e0698bf388/external/com_google_protobuf/BUILD", line 599
internal_gen_well_known_protos_java(srcs = WELL_KNOWN_PROTOS)
File "/root/.cache/bazel/_bazel_root/b6bc0828d5b43c7085b650e0698bf388/external/com_google_protobuf/protobuf.bzl", line 266, in internal_gen_well_known_protos_java
Label(("%s//protobuf_java" % REPOSITOR...))
File "/root/.cache/bazel/_bazel_root/b6bc0828d5b43c7085b650e0698bf388/external/com_google_protobuf/protobuf.bzl", line 266, in Label
REPOSITORY_NAME
builtin variable 'REPOSITORY_NAME' is referenced before assignment.
ERROR: /root/.cache/bazel/_bazel_root/b6bc0828d5b43c7085b650e0698bf388/external/com_google_protobuf/BUILD:382:1: Target '@com_google_protobuf//:android' contains an error and its package is in error and referenced by '@com_google_protobuf//:protoc'
ERROR: /root/.cache/bazel/_bazel_root/b6bc0828d5b43c7085b650e0698bf388/external/com_google_protobuf/BUILD:382:1: Target '@com_google_protobuf//:windows' contains an error and its package is in error and referenced by '@com_google_protobuf//:protoc'
ERROR: /root/.cache/bazel/_bazel_root/b6bc0828d5b43c7085b650e0698bf388/external/com_google_protobuf/BUILD:382:1: Target '@com_google_protobuf//:windows_msvc' contains an error and its package is in error and referenced by '@com_google_protobuf//:protoc'
ERROR: /root/.cache/bazel/_bazel_root/b6bc0828d5b43c7085b650e0698bf388/external/com_github_grpc_grpc/src/proto/grpc/reflection/v1alpha/BUILD:21:1: every rule of type proto_library implicitly depends upon the target '@com_google_protobuf//:protoc', but this target could not be found because of: Target '@com_google_protobuf//:protoc' contains an error and its package is in error
ERROR: Analysis of target '//tf_trusted:tf_trusted' failed; build aborted: Analysis failed
INFO: Elapsed time: 214.936s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (44 packages loaded, 2433 targets
FAILED: Build did NOT complete successfully (44 packages loaded, 2433 targets
configured)
currently loading: @org_tensorflow//tensorflow/contrib/lite/kernels
Fetching @boringssl; fetching 8s
Fetching @Jinja; fetching

The instructions to build and run tf-trusted in simulation mode crash with the following error on a machine without SGX:

2019-06-21 14:01:21  WARNING  untrusted_sgx.cc : 62 : Failed to create an enclave, attempt=0, status=1
2019-06-21 14:01:21  ERROR  enclave_manager.cc : 324 : LoadEnclave failed: ::asylo::error::SgxErrorSpace::Unexpected error: Failed to create an enclave
2019-06-21 14:01:21  ERROR  enclave_manager.cc : 324 : LoadEnclave failed: ::asylo::error::SgxErrorSpace::Unexpected error: Failed to create an enclave
2019-06-21 14:01:21  FATAL  tf_trusted_driver.cc : 54 : Load tf_trusted/tf_trusted_enclave.so failed: ::asylo::error::SgxErrorSpace::Unexpected error: Failed to create an enclave
2019-06-21 14:01:21  FATAL  tf_trusted_driver.cc : 54 : Load tf_trusted/tf_trusted_enclave.so failed: ::asylo::error::SgxErrorSpace::Unexpected error: Failed to create an enclave

Is this a requirement to have SGX even for the simulation mode?

We're currently not testing, linting or running the code through Continuous Integration. Once we have some linting or testing set up it should be fairly straight forward to set up CI.

Hi all,
I am trying to build TF Trusted Custom Operation. Starting the container using sudo docker run -it -v pwd:/opt/my-project \ -w /opt/my-project/tf_trusted_custom_op \ tensorflow/tensorflow:custom-op /bin/bash command gives the following error:

Collecting tensorflow==1.13.1
/usr/local/lib/python2.7/dist-packages/pip-9.0.3-py2.7.egg/pip/vendor/urllib3/util/ssl.py:339: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/pip-9.0.3-py2.7.egg/pip/vendor/urllib3/util/ssl.py:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
Could not find a version that satisfies the requirement tensorflow==1.13.1 (from versions: )
No matching distribution found for tensorflow==1.13.1
/usr/local/lib/python2.7/dist-packages/pip-9.0.3-py2.7.egg/pip/vendor/urllib3/util/ssl.py:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
Certificate did not match expected hostname: pypi.org. Certificate: {'notAfter': 'Nov 23 18:41:10 2022 GMT', 'subjectAltName': (('DNS', 'www.python.org'), ('DNS', '*.python.org'), ('DNS', 'docs.python.org'), ('DNS', 'downloads.python.org'), ('DNS', 'pypi.python.org')), 'subject': ((('commonName', u'www.python.org'),),)}
Traceback (most recent call last):
File "", line 1, in
ImportError: No module named 'tensorflow'
Traceback (most recent call last):
File "", line 1, in
ImportError: No module named 'tensorflow'

Does anyone know what the issue is and can help in resolving the error?
Thanks

In the following rules definitions (tf_trusted/BUILD), the rule name defined in sim_enclave is "tf_trusted_config_proto", and the dependencies in sim_enclave and enclave_loader is named "tf_trusted_config_proto_cc".

Just wanted to check if this is intended.

# Contains extensions to enclave protos.
asylo_proto_library(
    name = "tf_trusted_config_proto",
    srcs = ["tf_trusted_config.proto"],
    deps = ["@com_google_asylo//asylo:enclave_proto"],
)

# The enclave
sim_enclave(
    name = "tf_trusted_enclave.so",
    srcs = ["tf_trusted_enclave.cc"],
    config = "@com_google_asylo//asylo/grpc/util:grpc_enclave_config",
    deps = [
        ":tf_trusted_config_proto_cc",
        ":model_server",
        "@com_github_grpc_grpc//:grpc++",
        "@com_github_grpc_grpc//:grpc++_reflection",
        "@com_google_absl//absl/memory",
        "@com_google_absl//absl/synchronization",
        "@com_google_asylo//asylo:enclave_runtime",
        "@com_google_asylo//asylo/util:status",
    ],
)

# The driver for the gRPC server enclave.
enclave_loader(
    name = "tf_trusted",
    srcs = ["tf_trusted_driver.cc"],
    enclaves = {"enclave": ":tf_trusted_enclave.so"},
    loader_args = ["--enclave_path='{enclave}'"],
    deps = [
        ":tf_trusted_config_proto_cc",
        "@com_github_gflags_gflags//:gflags_nothreads",
        "@com_google_absl//absl/synchronization",
        "@com_google_absl//absl/time",
        "@com_google_asylo//asylo:enclave_client",
        "@com_google_asylo//asylo/util:logging",
    ],
)

Asylo 0.4.0 was recently released we should upgrade to his latest version. Could be as simple as updating the WORKSPACE file but I imagine something else will be needed.

Hi, I am trying to figure out how to perform model inference inside the enclave. I read the tf_trusted_enclave part, but it seems currently there is only a grpc server doing nothing. Do you have any suggestions on how to integrate the computation of model prediction into enclave as well as take into remote attestation into consideration? Thanks and best regards.

I was building the custom op using Docker. This is what I got.

INFO: SHA256 (https://boringssl.googlesource.com/boringssl/+archive/afc30d43eef92979b05776ec0963c9cede5fb80f.tar.gz) = e2e87a32861bceee0d5508e65175da756116ff4c8e1e16288a136d24146b515f
WARNING: /root/.cache/bazel/_bazel_root/6a072cedc59c5d9384722d447b964014/external/local_config_tf/BUILD:5:12: in hdrs attribute of cc_library rule @local_config_tf//:tf_header_lib: file '_api_implementation.so' from target '@local_config_tf//:tf_header_include' is not allowed in hdrs
WARNING: /root/.cache/bazel/_bazel_root/6a072cedc59c5d9384722d447b964014/external/local_config_tf/BUILD:5:12: in hdrs attribute of cc_library rule @local_config_tf//:tf_header_lib: file '_message.so' from target '@local_config_tf//:tf_header_include' is not allowed in hdrs
INFO: Analysed target //:model_enclave_op.so (21 packages loaded).
INFO: Found 1 target...
INFO: From SkylarkAction external/com_github_grpc_grpc/src/proto/grpc/reflection/v1alpha/reflection.pb.h:
bazel-out/k8-fastbuild/genfiles/external/com_github_grpc_grpc/external/com_github_grpc_grpc: warning: directory does not exist.
INFO: From SkylarkAction external/com_github_grpc_grpc/src/proto/grpc/reflection/v1alpha/reflection.grpc.pb.h:
bazel-out/k8-fastbuild/genfiles/external/com_github_grpc_grpc/external/com_github_grpc_grpc: warning: directory does not exist.
ERROR: /opt/my-project/tf_trusted_custom_op/BUILD:17:1: C++ compilation of rule '//:model_enclave_op.so' failed (Exit 1): gcc failed: error executing command 
  (cd /root/.cache/bazel/_bazel_root/6a072cedc59c5d9384722d447b964014/execroot/tf_trusted_custom_op && \
  exec env - \
    PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
    PWD=/proc/self/cwd \
    TF_HEADER_DIR=/usr/local/lib/python2.7/dist-packages/tensorflow/include \
    TF_SHARED_LIBRARY_DIR=/usr/local/lib/python2.7/dist-packages/tensorflow \
  /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -B/usr/bin -B/usr/bin -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer '-std=c++0x' -MD -MF bazel-out/k8-fastbuild/bin/_objs/model_enclave_op.so/model_enclave_op.pic.d '-frandom-seed=bazel-out/k8-fastbuild/bin/_objs/model_enclave_op.so/model_enclave_op.pic.o' -fPIC '-DPB_FIELD_32BIT=1' -DCARES_STATICLIB -iquote . -iquote bazel-out/k8-fastbuild/genfiles -iquote external/local_config_tf -iquote bazel-out/k8-fastbuild/genfiles/external/local_config_tf -iquote external/bazel_tools -iquote bazel-out/k8-fastbuild/genfiles/external/bazel_tools -iquote external/com_github_grpc_grpc -iquote bazel-out/k8-fastbuild/genfiles/external/com_github_grpc_grpc -iquote external/com_github_madler_zlib -iquote bazel-out/k8-fastbuild/genfiles/external/com_github_madler_zlib -iquote external/com_github_nanopb_nanopb -iquote bazel-out/k8-fastbuild/genfiles/external/com_github_nanopb_nanopb -iquote external/com_github_cares_cares -iquote bazel-out/k8-fastbuild/genfiles/external/com_github_cares_cares -iquote external/boringssl -iquote bazel-out/k8-fastbuild/genfiles/external/boringssl -iquote external/com_google_protobuf -iquote bazel-out/k8-fastbuild/genfiles/external/com_google_protobuf -isystem external/local_config_tf/include -isystem bazel-out/k8-fastbuild/genfiles/external/local_config_tf/include -isystem bazel-out/k8-fastbuild/bin/external/local_config_tf/include -isystem external/com_github_grpc_grpc/include -isystem bazel-out/k8-fastbuild/genfiles/external/com_github_grpc_grpc/include -isystem bazel-out/k8-fastbuild/bin/external/com_github_grpc_grpc/include -isystem external/com_github_madler_zlib -isystem bazel-out/k8-fastbuild/genfiles/external/com_github_madler_zlib -isystem bazel-out/k8-fastbuild/bin/external/com_github_madler_zlib -isystem external/com_github_cares_cares -isystem bazel-out/k8-fastbuild/genfiles/external/com_github_cares_cares -isystem bazel-out/k8-fastbuild/bin/external/com_github_cares_cares -isystem external/com_github_grpc_grpc/third_party/address_sorting/include -isystem bazel-out/k8-fastbuild/genfiles/external/com_github_grpc_grpc/third_party/address_sorting/include -isystem bazel-out/k8-fastbuild/bin/external/com_github_grpc_grpc/third_party/address_sorting/include -isystem external/boringssl/src/include -isystem bazel-out/k8-fastbuild/genfiles/external/boringssl/src/include -isystem bazel-out/k8-fastbuild/bin/external/boringssl/src/include -isystem external/com_google_protobuf/src -isystem bazel-out/k8-fastbuild/genfiles/external/com_google_protobuf/src -isystem bazel-out/k8-fastbuild/bin/external/com_google_protobuf/src -isystem cpp_model_proto_pb -isystem bazel-out/k8-fastbuild/genfiles/cpp_model_proto_pb -isystem bazel-out/k8-fastbuild/bin/cpp_model_proto_pb -pthread '-std=c++11' '-D_GLIBCXX_USE_CXX11_ABI=0' -fno-canonical-system-headers -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -c model_enclave_op.cc -o bazel-out/k8-fastbuild/bin/_objs/model_enclave_op.so/model_enclave_op.pic.o)
In file included from model_enclave_op.cc:2:0:
model_enclave_grpc.h: In member function 'bool ModelClient::GetModelPredict(std::string, const float*, int, T*, int)':
model_enclave_grpc.h:114:103: error: no matching function for call to 'google::protobuf::RepeatedField<long int>::ExtractSubrange(int, int, tensorflow::int64*)'
                 res.mutable_int64_result()->ExtractSubrange(0, res.float_result_size(), (int64*)output);
                                                                                                       ^
model_enclave_grpc.h:114:103: note: candidate is:
In file included from bazel-out/k8-fastbuild/genfiles/external/local_config_tf/include/google/protobuf/wire_format_lite_inl.h:44:0,
                 from bazel-out/k8-fastbuild/genfiles/external/local_config_tf/include/google/protobuf/map_type_handler.h:35,
                 from bazel-out/k8-fastbuild/genfiles/external/local_config_tf/include/google/protobuf/map.h:49,
                 from bazel-out/k8-fastbuild/genfiles/external/local_config_tf/include/google/protobuf/generated_message_table_driven.h:34,
                 from bazel-out/k8-fastbuild/genfiles/external/local_config_tf/include/tensorflow/core/framework/op_def.pb.h:25,
                 from bazel-out/k8-fastbuild/genfiles/external/local_config_tf/include/tensorflow/core/framework/op_def_builder.h:24,
                 from bazel-out/k8-fastbuild/genfiles/external/local_config_tf/include/tensorflow/core/framework/op.h:23,
                 from model_enclave_grpc.h:1,
                 from model_enclave_op.cc:2:
bazel-out/k8-fastbuild/genfiles/external/local_config_tf/include/google/protobuf/repeated_field.h:1204:6: note: void google::protobuf::RepeatedField<Element>::ExtractSubrange(int, int, Element*) [with Element = long int] <near match>
 void RepeatedField<Element>::ExtractSubrange(
      ^
bazel-out/k8-fastbuild/genfiles/external/local_config_tf/include/google/protobuf/repeated_field.h:1204:6: note:   no known conversion for argument 3 from 'tensorflow::int64* {aka long long int*}' to 'long int*'
Target //:model_enclave_op.so failed to build
INFO: Elapsed time: 277.132s, Critical Path: 21.07s
INFO: 1066 processes: 1066 local.
FAILED: Build did NOT complete successfully

Whenever exits (e.g. print statements) or other system calls happen it causes the SGX program to be slower than it needs to be. We can audit the code to make sure we remove all prints and other obvious system calls. To discover system calls that might be occurring that we're not aware of, inside TF Lite for example, we can use strace to investigate further. Here's a quick tutorial explaining how to use strace https://www.youtube.com/watch?v=EG0ihttnEJI.

Another route to take is to look deeper into sgx switchless code. This makes it so that CPU doesn't need to switch from enclave mode to unprotected mode when calling exits or system calls. Figuring out if Asylo supports this by default would be a good place to start. More information on switchless in SGX can be found here https://github.com/intel/linux-sgx/tree/master/SampleCode/Switchless and here https://github.com/intel/linux-sgx/tree/master/sdk/switchless.

Remote Attestation allows an SGX/enclave device to prove that it is a trusted device. Currently Asylo does not have remote attestation implemented. Further details here.

Another route we could take is to implement Intel Remote Attestation ourselves using this guide and extending the Asylo interface suggested in the above asylo thread.

TF-lite can only do the inference, which means if I want to do training inside SGX, I have to use the full version of the TensorFlow?

Ran MNIST, but running Deep Learning Models (MobileNet for ex.) fails because of the limited memory. What approaches are you guys seeking to enable heavier computations ?
Outsourcing the Compute to a GPU as described in Slalom ? or keep it in the CPU and release neural network layers in chunks to the TEE as described in ML Capsule?
We are currently experimenting with chunking layers and sending to the enclave in batches and processing them. Let me know your thoughts

Hello,

I have an issue when trying to build and run TF Trusted on the SGX Device. When working in the simulation mode everything works just fine, but when trying to build the docker pointing to the SGX Device, I get following error:

ERROR: Unrecognized option: --incompatible_disallow_filetype=false

Any help would be very very much appreciated!