stanford-futuredata / noscope

Accelerating network inference over video

NoScope

This is the official project page for the NoScope project.

Please read the blog post and paper for more details!

Requirements

This repository contains the code for the optimization step in the paper. The inference code is here.

You will need the following installed:

• python 2.7
• pip python-setuptools python-tk
• keras
• CUDA, CUDNN, tensorflow-gpu
• OpenCV 3.2 with FFmpeg bindings
• g++ 5.4 or later

Your machine will need at least:

• AVX2 capabilities
• 300+GB of memory
• 500+GB of space
• A GPU (this has only been tested with NVIDIA K80 and P100)

Guides on Installing the Requirements

• python 2.7 - For Linux, use your package manager. For Mac: http://docs.python-guide.org/en/latest/starting/install/osx/.
• pip python-setuptools python-tk - https://packaging.python.org/tutorials/installing-packages/
• Install version 1.2 of keras, as it must interface with TensorFlow 0.12 - pip install keras==1.2
• CUDA, CUDNN, tensorflow-gpu
  • https://www.tensorflow.org/versions/r0.12/get_started/os_setup - YOU MUST INSTALL THESE VERSIONS, not more recent ones. Go to the section Download and Install cuDNN for cuDNN installation instructions.
  • Note: Having both TensorFlow-gpu 0.12 and more recent versions installed is complicated. It may not be easy to have the multiple versions of cuDNN installed. This project requires cuDNN 5.1 and more recent versions of TensorFlow will break with that installed. Therefore, it is recommended that users uninstall more recent versions of tensorflow and delete the cuDNN so files if they are installed.
• OpenCV 3.2 with FFmpeg bindings - https://github.com/BVLC/caffe/wiki/OpenCV-3.2-Installation-Guide-on-Ubuntu-16.04
• g++ 5.4 or later - For Linux, use your package manager. For Mac, http://braumeister.org/formula/gcc@5 should work, though the developers haven't tested this.

Setting up the inference engine

To set up the inference engine, do the following: Note: It is recommended that you create a folder that contains this repository, the tensorflow-noscope repository, and the data folder referred to below.

git clone https://github.com/stanford-futuredata/tensorflow-noscope.git
cd tensorflow-noscope
git checkout speedhax
git submodule init
git submodule update
./configure
cd tensorflow
bazel build -c opt --copt=-mavx2 --config=cuda noscope

The build will fail. To fix this, update the BUILD file to point towards your OpenCV install and add this directory to your PATH environmental variable. The BUILD file is in the tensorflow-noscope git repository at tensorflow/noscope/BUILD. You will need to change all references to "/lfs/0/ddkang/". You will probably need to change these to "/usr/" if you installed OpenCV using the directions above.

Please encourage the Tensorflow developers to support non-bazel building and linking. Due to a quirk in bazel, it may occasionally "forget" that tensorflow-noscope was built. If this happens, rebuild.

Setting up the optimization engine

To set up the optimization engine, install the NoScope python package by going to the root directory of where you checked out https://github.com/stanford-futuredata/noscope and running "pip install -e ./"

Running the example

Once you have inference engine set up, the example/ subfolder within this repository contains the script to reproduce Figure 5d in the paper.

In order to run this:

1. Create a folder named data that sits in the same directory as your noscope and tensorflow-noscope folders
2. Create the following folders within the data folder: videos, csv, cnn-avg, cnn-models, and experiments
3. Download the coral-reef video and labels, putting the csv file in the csv folder and the mp4 file in the videos folder:

wget https://storage.googleapis.com/noscope-data/csvs-yolo/coral-reef-long.csv
wget https://storage.googleapis.com/noscope-data/videos/coral-reef-long.mp4

4. Update the code and data paths in example/run.sh. code should point to the folder that contains both the noscope and tensorflow-noscope folders. This value is how the optimization and inference engines find eachother. data should point to the data folder created in this section.
5. Download the YOLO neural network weights file from https://pjreddie.com/media/files/yolo.weights. It is suggested that you place the file at the location tensorflow-noscope/tensorflow/noscope/darknet/weights/. Note that you will need to make the weights folder.
6. Update example/noscope_motherdog.py to point to the YOLO configuration and weights files. The config file is tensorflow-noscope/tensorflow/noscope/darknet/cfg/yolo.cfg and the weights file is the one you downloaded. If you put the weights file in the suggested location, this step should be unnecessary.
7. Run example/run.sh. The outputted summary.csv will be in the location data/experiments/$VIDEO_NAME.

Datasets

The datasets that are currently available are coral-reef-long and jackson-town-square. Due to the expense of hosting these files, we have turned on requester pays for download. Please use an authenticated gsutil to download the files.

The mp4 video files are available at https://storage.googleapis.com/noscope-data/videos/VIDEO_NAME.mp4

The CSVs with ground truth labels are available at https://storage.googleapis.com/noscope-data/csvs-yolo/VIDEO_NAME.csv

If the above links do not work, you can download the data on Google drive here.

noscope's Issues

Video camera source

Hi, all
It seems that the link for the video is unavailable at this time. Can anybody send me a new link about the video source?
Many thanks!!

What the protobuf version the project it need?

rt, can you tell me what the protobuf version it need?Thank you very much.

fatal: reference is not a tree: d3ea51f6bbe3e20d37b5b449f15903f2a5a7a37d

when I do "git submodule update", I get this error:
fatal: reference is not a tree: d3ea51f6bbe3e20d37b5b449f15903f2a5a7a37d
Unable to checkout 'd3ea51f6bbe3e20d37b5b449f15903f2a5a7a37d' in submodule path 'tensorflow/noscope/darknet'

Could you please help me.

OSError: [Errno 2] No such file or directory: '/home/zhangrf/NOSCOPE/data/experiments/coral-reef-long'

when I run the run.sh i meet with the following issue:

which shows that there are some problems in noscope_motherdog.py , so i open the py and find the related code, as is following:

and i think the code tells that if i have no experiment_dir ,it will make it automatically, but the fact is that though it print the sentence as following:

but there is no directory generated in fact，so when it run the code "os.chdir( experiment_dir )", it raise the issue ,i am not sure how to debug it,can you help me?

i got one error about git darknet

hi all
i run : git submodule init git submodule update
but i got an error
fatal: reference is not a tree: d3ea51f6bbe3e20d37b5b449f15903f2a5a7a37d
Unable to checkout 'd3ea51f6bbe3e20d37b5b449f15903f2a5a7a37d' in submodule path 'tensorflow/noscope/darknet'
no any code in darknet
thank your help

Can this system get detection rectangle for each frame?

I am not sure whether I fully digested this work. Is this work NoScope can replace YOLO v2 but reach much higher speed? The most important question I am concerned is that, YOLO v2 can not only detect whether a frame have a bus, but also can give the position and rectangle where the bus is.
Can NoScope give the position and rectangle where the bus is? Or NoScope can only classify whether a frame have or not have a bus?
The detection position is very important for our lab, so we hope we can know whether NoScope can provide detection rectangles.
Thank you!

Python 3 Compatibility

It would be great if the code could be made py3 compatible. The current blockers are the print commands and the relative imports

More datasets ?

I am experimenting with different types of difference detectors that I would like to contribute to the noscope project. Is there a chance that you can share the rest of the 6 videos studied in the paper so that I can make comprehensive comparison between the difference detectors ?

300+GB of memory

I am confused how does this system can use up to 300+GB of memory ?

I got an error when setting up the inference engine

After I input the code "git submodule update", I got an error:

I think the path is correct.
Could you please help me ?

Use my own video to run NoScope

First my video TimeSquare.mp4 is 29 minutes, is that long enough for NoScope study?
How to create the TimeSquare.csv required by NoScope?

Thanks!

OpenCV Error：resize.cpp

hello there,
I was trying to reproduce the noscope and met the problem:
OpenCV Error: Assertion failed (ssize.width > 0 && ssize.height > 0) in resize, file /root/opencv/modules/imgproc/src/resize.cpp, line 3289
terminate called after throwing an instance of 'cv::Exception'
what(): /root/opencv/modules/imgproc/src/resize.cpp:3289: error: (-215) ssize.width > 0 && ssize.height > 0 in function resize

Any help?
Thanks in advance.

GTX1070:CUDA Error: out of memory

GTX1070：7.9G memory
when i run_optimizerset.sh,the train_9180_18360.log displayed errors.

train_9180_18360.log:
2017-08-28 17:31:50.229531: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
2017-08-28 17:31:50.351945: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2017-08-28 17:31:50.352224: I tensorflow/core/common_runtime/gpu/gpu_device.cc:887] Found device 0 with properties:
name: GeForce GTX 1070
major: 6 minor: 1 memoryClockRate (GHz) 1.8225
pciBusID 0000:01:00.0
Total memory: 7.92GiB
Free memory: 7.31GiB
2017-08-28 17:31:50.352235: I tensorflow/core/common_runtime/gpu/gpu_device.cc:908] DMA: 0
2017-08-28 17:31:50.352240: I tensorflow/core/common_runtime/gpu/gpu_device.cc:918] 0: Y
2017-08-28 17:31:50.352249: I tensorflow/core/common_runtime/gpu/gpu_device.cc:977] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1070, pci bus id: 0000:01:00.0)
layer filters size input output
0 CUDA Error: out of memory: File exists
CUDA Error: out of memory

Updated dataset link?

I am trying to download the dataset but the given link is not working for me? anyone has updated link. it would be nice to have it for experiment purpose.
Thank You

Error:
(tfg) [root@pinky csv]# wget https://storage.googleapis.com/noscope-data/videos/coral-reef-long.mp4
--2019-07-11 11:47:41-- https://storage.googleapis.com/noscope-data/videos/coral-reef-long.mp4
Resolving storage.googleapis.com (storage.googleapis.com)... 2a00:1450:4001:817::2010, 172.217.21.208
Connecting to storage.googleapis.com (storage.googleapis.com)|2a00:1450:4001:817::2010|:443... connected.
HTTP request sent, awaiting response... 400 Bad Request
2019-07-11 11:47:42 ERROR 400: Bad Request.

Run noscope got error

I have installed tensorflow-noscope and noscope by following the instruction. When I run
noscope/example/run.sh

It complained:

ImportError: No module named noscope
ImportError: No module named tensorflow

So what did I miss to have python finds noscope and tensorflow?

Thanks,
Jan

I can't find a way to produce the train_{Start_index}_{end_index}.csv

As the training file is required in noscope_optimizer.py, I can not find a way to produce the file. Can you please give me some hints? Thank you

Closed: I found out that my GPU doesn't support AVX2

Access denied to noscope-data

Hi,

I tried to download the dataset using gsutil , below is the bash scripts I have.

gsutil -u downloadtest-249815 cp gs://noscope-data/csvs-yolo/coral-reef-long.csv . 

However, I got the following error message,

AccessDeniedException: 403 [email protected] does not have storage.objects.list access to noscope-data.

I wonder is there anything wrong with my method? Or do I need to first get permission from somewhere else before downloading the related dataset?

Thanks,
Bo

Visualizer.py - TypeError: string indices must be integers, not str

When I run Visualizer.py for (jackson-town-square.mp4 + jackson-town-square.csv), the script throws an error:

Traceback (most recent call last):
  File "/opt/pycharm-2017.2.2/helpers/pydev/pydevd.py", line 1596, in <module>
    globals = debugger.run(setup['file'], None, None, is_module)
  File "/opt/pycharm-2017.2.2/helpers/pydev/pydevd.py", line 1023, in run
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/home/user/noscope/noscope/Visualizer.py", line 81, in <module>
    main()
  File "/home/user/noscope/noscope/Visualizer.py", line 77, in main
    draw_video(args.input_csv, args.video_in, nb_frames, args.start_frame, objects)
  File "/home/user/noscope/noscope/Visualizer.py", line 56, in draw_video
    if label['object_name'] not in OBJECTS:
TypeError: string indices must be integers, not str

The code in that place looks strange as you're trying to enumerate the pandas dataframe instead of df.iterrows():

    for i, labels in enumerate(all_labels):
        if i % 500 == 0:
            print i
        for label in labels:
            if label['object_name'] not in OBJECTS:
                continue
            draw_label(label, frame, obj_to_color[label['object_name']])
        vout.write(frame)
        ret, frame = cap.read()

As a result in for loop the variable labels is a string and is equal to "object_name" and the further iteration over it iterates over the string.

pandas version: 0.20.3
python version: 2.7.6

Analysis of target '//tensorflow/noscope:noscope' failed; build aborted.

when i run the command: bazel build -c opt --copt=-mavx2 --config=cuda noscope,i meet the issue,and i do not know how to fix it ,can you help me?thanks!

2 errors detected in the compilation of "/tmp/tmpxft_00016431_00000000-10_yolo.cu.compute_52.cpp1.ii".

I’m having the following issue when running the bazel build -c opt --copt=-mavx2 --config=cuda noscope
INFO: From Compiling tensorflow/noscope/darknet/src/yolo.cu.cc:
tensorflow/noscope/darknet/src/yolo.cu.cc(36): error: namespace "std" has no member "runtime_error"

tensorflow/noscope/darknet/src/yolo.cu.cc(57): error: namespace "std" has no member "runtime_error"

2 errors detected in the compilation of "/tmp/tmpxft_00016431_00000000-10_yolo.cu.compute_52.cpp1.ii".
ERROR: /home/five/PycharmProjects/chenglin/dataset/IPIU/tensorflow-noscope/tensorflow/noscope/darknet/BUILD:10:1: output 'tensorflow/noscope/darknet/_objs/yolo_gpu/tensorflow/noscope/darknet/src/yolo.cu.o' was not created.
ERROR: /home/five/PycharmProjects/chenglin/dataset/IPIU/tensorflow-noscope/tensorflow/noscope/darknet/BUILD:10:1: not all outputs were created or valid.
Target //tensorflow/noscope:noscope failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 8.297s, Critical Path: 7.96s

Has anyone seen this error before? Please help me,Thank you

VUSELabeler.py - AttributeError: 'module' object has no attribute 'YOLO'

When I run python VUSELabeler.py --yolo_dir /path/to/darkflow --output_csv_fname /path/to/output.csv --input_video_fname /path/to/output.avi --start_from 0 it throws an error:

Using TensorFlow backend.
Traceback (most recent call last):
  File "VUSELabeler.py", line 85, in <module>
    main()
  File "VUSELabeler.py", line 79, in main
    args.yolo_dir, args.start_from)
  File "VUSELabeler.py", line 38, in label_video
    yolo_dir + '/cfg/coco.data')
  File "VUSELabeler.py", line 15, in __init__
    self.YOLO = noscope.YOLO(config_filename, weights_filename, data_config)
AttributeError: 'module' object has no attribute 'YOLO'

Error in bazel build

Unable to build using bazel
While running
bazel build -c opt --copt=-mavx2 --config=cuda --verbose_failures --cxxopt="-D_GLIBCXX_USE_CXX11_ABI=0" noscope

Error : js_embed: symbol _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE9_M_createERmm, version GLIBCXX_3.4.21 not defined in file libstdc++.so.6 with link time reference

when demo out？

This job is very great， and the paper also enlightening！！
I want to try it, but I don`t know how to run it. Could you update some demo or illustrate for this?
thank you

IOError: [Errno 2] No such file or directory

when I run the run.sh, after 'Generate the models', 'Convert keras models to TF models', 'Generate the avg' were done and the 'Run the inference' was just start, i meet with the following issue

Could you let me know the 'train_918000_1836000.csv' was created in which step or place? And what's the possible reasons that created this problem, because the several steps before that were executed successfully. Thank you.

Is this project under developing or ready to use?

Hi all

Just found the repo from the original publication. I am wondering if this project is ready to test or demo. It seems like there is no documentation and there are no commits in the last four months.

great work！documentation please

read the noscope article it's a good idea to add a difference detector on top of Object Detector since my project uses the scene cut detection to reduce the computation but on the other hand losing many important frames.

Downloading dataset

I am trying to use GCP and gsutil to download the dataset but I couldn't find any instructions on how to do that. I already enabled the billing account on GCP and when I am trying this command gsutil -u enduring-maxim-269304 cp gs://noscope-data/csvs-yolo/coral-reef-long.csv ~/Downloads/noscope-data, it gives me this error 'AccessDeniedException: 403 [email protected] does not have storage.objects.list access to noscope-data'. Can you please provide some basic instructions on how to download them.

Bests
Thanks

ValueError: Input 0 is incompatible with layer conv2d_1: expected ndim=4, found ndim=3

when i run the ./run.sh
it appers that

I think that our input's ndim should be 3 but it just tell me it expected 4 ,and throw the error , i do not know what to do
what is more, even i have comment the coral-reef-long part in run.sh
when i run the script,it still appear like this:

please help me!

How to run the demo?

Can you provide the instructions on how to run the demo include train and detect?

Very interesting approach. Is it possible to share the demo that you worked on the paper?

I am very interested in this approach. Is it possible to share the demo which you did on the paper?

Is opencv 3.4 ok to reproduce noscope?

hi there! Thank you for your great work in this detection mission. It was inspiring. I was trying to reproduce it. I followed every step but the opencv version. Could it be the reason for the resize error ?
cv2.error: OpenCV(3.4.3) /io/opencv/modules/imgproc/src/resize.cpp:4044: error: (-215:Assertion failed) !ssize.empty() in function 'resize'

SIFT difinition no SIFT.get_distance_fn

def get_feature_and_dist_fns(feature_type):
if feature_type == 'hog':
return (HOG.compute_feature, HOG.get_distance_fn, HOG.DIST_METRICS)
elif feature_type == 'sift':
return (SIFT.compute_feature, SIFT.get_distance_fn, SIFT.DIST_METRICS)
elif feature_type == 'ch':
return (ColorHistogram.compute_feature, ColorHistogram.get_distance_fn, ColorHistogram.DIST_METRICS)
elif feature_type == 'raw':
return (RawImage.compute_feature, RawImage.get_distance_fn, RawImage.DIST_METRICS)

import cv2
import numpy as np
from scipy.spatial.distance import euclidean, cityblock, chebyshev, cosine

DIST_METRICS = [
('euclidean', euclidean),
('manhattan', cityblock),
('chebyshev', chebyshev),
('cosine', lambda x, y: -1*cosine(x, y)),
#('chisqr', lambda x, y: cv2.compareHist(x, y, cv2.HISTCMP_CHISQR)),
#('bhatta', lambda x, y: cv2.compareHist(x, y, cv2.HISTCMP_BHATTACHARYYA))
]

def compute_feature(frame):
sift = cv2.xfeatures2d.SIFT_create()
image = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
_, des = sift.detectAndCompute(image, None)
if des is not None:
return np.mean(des, axis=0).astype('float32')
else:
return np.zeros(128)

SIFT difinition no SIFT.get_distance_fn

Any visualizing script for video files?

I was looking into the scripts and hope that I can find there something that could help to visualize testing results (draw bounding boxes on frames just like in the article on that GIFs animations), but haven't found it. Is there such a tool in the source code or not?

fail to download the dataset

<Code>AccessDenied</Code>
<Message>Access denied.</Message>
<Details>
Anonymous users does not have storage.objects.get access to noscope-data/videos/VIDEO_NAME.mp4.
</Details>
</Error>

Could I obtain the data set by other means?

/usr/local/lib/libopencv_core.so.3.2:error adding symbols:DSO missing from command line

when i use the command bazel build -c opt --copt=-mavx2 --config=cuda noscope
and everything is fine but i meet this issue

and then i use the command bazel build -c opt --copt=-mavx2 --config=cuda noscope -lopencv_core and have no command,i don not know how ti fix it
when i use the command ./configure
i decide not to build TensorFlow with OpenCL support,i do not know if it matters.

To understand NoScope

I finally got NoScope running on my Linux with 2 GPUs. To help me understand NoScope better, I have the following questions:

1. For NoScope results, what do those folders "0.0", "0.01", "0.02", ..."0.1", "0.25" mean? How to understand summary.csv?
2. Is the small specialized model the same NN as the reference model but just with less layers? Does NoScope uses any of the traditional model from OpenCV?
3. What is NoScope's software architecture?
4. How to test the optimized performance improvement over reference model, or over the traditional OpenCV?
5. Is there a way to adjust the optimization engine? For instance, can I turn off the optimization?

Thanks!
Jan

Support Yolo3 and CPU?

Hi, this work is so attractive in video detection mission. Just want to know if this code can be used in CPU? Plus, can I simply add the yolo3 module in this code?

Thank you!

How to prepare CSV with ground truth labels for new video

I noticed that NoScope needs CSV with ground truth labels along with the video as input.
Two questions:

1. Is the ground truth for the entire video or just for the beginning part of the video? If it is only for partial video, what is that partial, for example, the first half of the video or first 20% of the video (in time) or a percentage like that? That will help me to prepare my CSV.

2. If the ground truth is for the entire video, does NoScope really use all the ground truth? If so, what is the purpose of processing if all the ground truth is known as input?

Thanks!
Jan

what bazel version did you guys use to build?

I was having problems with both 0.5.4 which is what I guess the authors used back then and 0.10.0. They have different problems.

Just wondering which version did you guys use and successfully build it?

AttributeError: 'module' object has no attribute 'YOLO'

when i run VUSELabeler.py，it says no YOLO attribut, it seems that the YOLO class is missing under noscop dir:

ubuntu@XP1:~/cvs/noscope$ python scripts/VUSELabeler.py Using TensorFlow backend. I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcublas.so.8.0 locally I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcudnn.so.5 locally I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcufft.so.8.0 locally I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcuda.so.1 locally I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcurand.so.8.0 locally Traceback (most recent call last): File "scripts/VUSELabeler.py", line 85, in <module> main() File "scripts/VUSELabeler.py", line 79, in main args.yolo_dir, args.start_from) File "scripts/VUSELabeler.py", line 38, in label_video yolo_dir + '/cfg/coco.data') File "scripts/VUSELabeler.py", line 15, in __init__ self.YOLO = noscope.YOLO(config_filename, weights_filename, data_config) AttributeError: 'module' object has no attribute 'YOLO'

I’m interested in this great work，and can you give me a README?

Thanks very much!

Few questions

Hi Daniel,

I am really interested in integrating your system into Deep Video Analytics by combining the difference detector with segmented video to make the system a lot more memory efficient. I had few questions:

1. How long does it takes to train the frame-difference -> label model?

2. Is this model trained continuously or is it trained once per video stream/background?

3. Can this system be used for video feeds where background is NOT constant?

4. Is there any Open Source license associated with the codebase?

Thanks,

Why is it called NoScope?

Hi all:
After reading the paper, I still don't know why this framework is called NoScope.
Is it some kind of abbreviation?

Can I output the optimized model?

Once NoScope optimized the model for a video, can it output the optimized model so I can use it to process more videos from the same fixed camera without having to optimize again for each video? For instance, I have 5 videos from the same camera. NoScope found the optimized model from the 1st video, then how to run that model for the remaining 4 videos without the optimization again? Thank you!

AccessDeniedException: 403 The project to be billed is associated with a closed billing account.

I tried to download the dataset using gsutil
I have opened a new google cloud account. And I can create a bucket with the command:
gsutil -u download-255905 mb gs://wtx8887-download/

However, when I want to download the dataset, (using the bash scripts like #47 )
gsutil -u download-255905 cp gs://noscope-data/csvs-yolo/coral-reef-long.csv .

I faced issue as below,
AccessDeniedException: 403 The project to be billed is associated with a closed billing account.

I checked my account, its Statu was Active.
I wonder that is there anything wrong with my method?

Thanks.

Target //tensorflow/noscope:noscope failed to build

when i run

bazel build -c opt --copt=-mavx2 --config=cuda noscope

i got some error

ERROR: /home/user08/video_object_detection/noscope/tensorflow-noscope/tensorflow/stream_executor/BUILD:39:1: C++ compilation of rule '//tensorflow/stream_executor:cuda_platform' failed (Exit 1).
tensorflow/stream_executor/cuda/cuda_dnn.cc: In instantiation of 'cudnnStatus_t perftools::gputools::cuda::wrap::WrapperShim__cudnnSetRNNDescriptor::operator()(perftools::gputools::cuda::CUDAExecutor*, Args ...) [with Args = {cudnnRNNStruct*, int, int, cudnnDropoutStruct*, cudnnRNNInputMode_t, cudnnDirectionMode_t, cudnnRNNMode_t, cudnnDataType_t}]':
tensorflow/stream_executor/cuda/cuda_dnn.cc:1017:50: required from here
tensorflow/stream_executor/cuda/cuda_dnn.cc:139:38: error: cannot convert 'cudnnRNNStruct*' to 'cudnnHandle_t {aka cudnnContext*}' for argument '1' to 'cudnnStatus_t cudnnSetRNNDescriptor(cudnnHandle_t, cudnnRNNDescriptor_t, int, int, cudnnDropoutDescriptor_t, cudnnRNNInputMode_t, cudnnDirectionMode_t, cudnnRNNMode_t, cudnnRNNAlgo_t, cudnnDataType_t)'
cudnnStatus_t retval = ::__name(args...);
^
tensorflow/stream_executor/cuda/cuda_dnn.cc:233:3: note: in expansion of macro 'PERFTOOLS_GPUTOOLS_CUDNN_WRAP'
__macro(cudnnSetRNNDescriptor)
^
tensorflow/stream_executor/cuda/cuda_dnn.cc:238:1: note: in expansion of macro 'CUDNN_DNN_ROUTINE_EACH_R5'
CUDNN_DNN_ROUTINE_EACH_R5(PERFTOOLS_GPUTOOLS_CUDNN_WRAP)
^
In file included from tensorflow/stream_executor/cuda/cuda_dnn.cc:42:0:
bazel-out/local_linux-opt/genfiles/external/local_config_cuda/cuda/cuda/include/cudnn.h:1507:8: note: class type 'cudnnRNNStruct' is incomplete
struct cudnnRNNStruct;
^
can someone help me fix this?

ERROR: /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/noscope/darknet/BUILD:10:1:

when i run

bazel build -c opt --copt=-mavx2 --config=cuda noscope

i got some error

ERROR: /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/noscope/darknet/BUILD:10:1: in srcs attribute of cc_library rule //tensorflow/noscope/darknet:yolo_gpu: file '//tensorflow/noscope/darknet:src/activation_kernels.cu' is misplaced here (expected .cc, .cpp, .cxx, .c++, .C, .c, .h, .hh, .hpp, .hxx, .inc, .S, .s, .asm, .a, .lib, .pic.a, .lo, .pic.lo, .so, .dylib, .dll, .o or .pic.o). Since this rule was created by the macro 'tf_kernel_library', the error might have been caused by the macro implementation in /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/tensorflow.bzl:634:12.
ERROR: /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/noscope/darknet/BUILD:10:1: in srcs attribute of cc_library rule //tensorflow/noscope/darknet:yolo_gpu: '//tensorflow/noscope/darknet:src/activation_kernels.cu' does not produce any cc_library srcs files (expected .cc, .cpp, .cxx, .c++, .C, .c, .h, .hh, .hpp, .hxx, .inc, .S, .s, .asm, .a, .lib, .pic.a, .lo, .pic.lo, .so, .dylib, .dll, .o or .pic.o). Since this rule was created by the macro 'tf_kernel_library', the error might have been caused by the macro implementation in /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/tensorflow.bzl:634:12.
ERROR: /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/noscope/darknet/BUILD:10:1: in srcs attribute of cc_library rule //tensorflow/noscope/darknet:yolo_gpu: file '//tensorflow/noscope/darknet:src/avgpool_layer_kernels.cu' is misplaced here (expected .cc, .cpp, .cxx, .c++, .C, .c, .h, .hh, .hpp, .hxx, .inc, .S, .s, .asm, .a, .lib, .pic.a, .lo, .pic.lo, .so, .dylib, .dll, .o or .pic.o). Since this rule was created by the macro 'tf_kernel_library', the error might have been caused by the macro implementation in /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/tensorflow.bzl:634:12.
ERROR: /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/noscope/darknet/BUILD:10:1: in srcs attribute of cc_library rule //tensorflow/noscope/darknet:yolo_gpu: '//tensorflow/noscope/darknet:src/avgpool_layer_kernels.cu' does not produce any cc_library srcs files (expected .cc, .cpp, .cxx, .c++, .C, .c, .h, .hh, .hpp, .hxx, .inc, .S, .s, .asm, .a, .lib, .pic.a, .lo, .pic.lo, .so, .dylib, .dll, .o or .pic.o). Since this rule was created by the macro 'tf_kernel_library', the error might have been caused by the macro implementation in /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/tensorflow.bzl:634:12.
ERROR: /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/noscope/darknet/BUILD:10:1: in srcs attribute of cc_library rule //tensorflow/noscope/darknet:yolo_gpu: file '//tensorflow/noscope/darknet:src/blas_kernels.cu' is misplaced here (expected .cc, .cpp, .cxx, .c++, .C, .c, .h, .hh, .hpp, .hxx, .inc, .S, .s, .asm, .a, .lib, .pic.a, .lo, .pic.lo, .so, .dylib, .dll, .o or .pic.o). Since this rule was created by the macro 'tf_kernel_library', the error might have been caused by the macro implementation in /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/tensorflow.bzl:634:12.
ERROR: /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/noscope/darknet/BUILD:10:1: in srcs attribute of cc_library rule //tensorflow/noscope/darknet:yolo_gpu: '//tensorflow/noscope/darknet:src/blas_kernels.cu' does not produce any cc_library srcs files (expected .cc, .cpp, .cxx, .c++, .C, .c, .h, .hh, .hpp, .hxx, .inc, .S, .s, .asm, .a, .lib, .pic.a, .lo, .pic.lo, .so, .dylib, .dll, .o or .pic.o). Since this rule was created by the macro 'tf_kernel_library', the error might have been caused by the macro implementation in /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/tensorflow.bzl:634:12.
ERROR: /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/noscope/darknet/BUILD:10:1: in srcs attribute of cc_library rule //tensorflow/noscope/darknet:yolo_gpu: file '//tensorflow/noscope/darknet:src/col2im_kernels.cu' is misplaced here (expected .cc, .cpp, .cxx, .c++, .C, .c, .h, .hh, .hpp, .hxx, .inc, .S, .s, .asm, .a, .lib, .pic.a, .lo, .pic.lo, .so, .dylib, .dll, .o or .pic.o). Since this rule was created by the macro 'tf_kernel_library', the error might have been caused by the macro implementation in /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/tensorflow.bzl:634:12.
ERROR: /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/noscope/darknet/BUILD:10:1: in srcs attribute of cc_library rule //tensorflow/noscope/darknet:yolo_gpu: '//tensorflow/noscope/darknet:src/col2im_kernels.cu' does not produce any cc_library srcs files (expected .cc, .cpp, .cxx, .c++, .C, .c, .h, .hh, .hpp, .hxx, .inc, .S, .s, .asm, .a, .lib, .pic.a, .lo, .pic.lo, .so, .dylib, .dll, .o or .pic.o). Since this rule was created by the macro 'tf_kernel_library', the error might have been caused by the macro implementation in /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/tensorflow.bzl:634:12.
ERROR: /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/noscope/darknet/BUILD:10:1: in srcs attribute of cc_library rule //tensorflow/noscope/darknet:yolo_gpu: file '//tensorflow/noscope/darknet:src/convolutional_kernels.cu' is misplaced here (expected .cc, .cpp, .cxx, .c++, .C, .c, .h, .hh, .hpp, .hxx, .inc, .S, .s, .asm, .a, .lib, .pic.a, .lo, .pic.lo, .so, .dylib, .dll, .o or .pic.o). Since this rule was created by the macro 'tf_kernel_library', the error might have been caused by the macro implementation in /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/tensorflow.bzl:634:12.
ERROR: /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/noscope/darknet/BUILD:10:1: in srcs attribute of cc_library rule //tensorflow/noscope/darknet:yolo_gpu: '//tensorflow/noscope/darknet:src/convolutional_kernels.cu' does not produce any cc_library srcs files (expected .cc, .cpp, .cxx, .c++, .C, .c, .h, .hh, .hpp, .hxx, .inc, .S, .s, .asm, .a, .lib, .pic.a, .lo, .pic.lo, .so, .dylib, .dll, .o or .pic.o). Since this rule was created by the macro 'tf_kernel_library', the error might have been caused by the macro implementation in /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/tensorflow.bzl:634:12.
ERROR: /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/noscope/darknet/BUILD:10:1: in srcs attribute of cc_library rule //tensorflow/noscope/darknet:yolo_gpu: file '//tensorflow/noscope/darknet:src/crop_layer_kernels.cu' is misplaced here (expected .cc, .cpp, .cxx, .c++, .C, .c, .h, .hh, .hpp, .hxx, .inc, .S, .s, .asm, .a, .lib, .pic.a, .lo, .pic.lo, .so, .dylib, .dll, .o or .pic.o). Since this rule was created by the macro 'tf_kernel_library', the error might have been caused by the macro implementation in /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/tensorflow.bzl:634:12.
ERROR: /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/noscope/darknet/BUILD:10:1: in srcs attribute of cc_library rule //tensorflow/noscope/darknet:yolo_gpu: '//tensorflow/noscope/darknet:src/crop_layer_kernels.cu' does not produce any cc_library srcs files (expected .cc, .cpp, .cxx, .c++, .C, .c, .h, .hh, .hpp, .hxx, .inc, .S, .s, .asm, .a, .lib, .pic.a, .lo, .pic.lo, .so, .dylib, .dll, .o or .pic.o). Since this rule was created by the macro 'tf_kernel_library', the error might have been caused by the macro implementation in /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/tensorflow.bzl:634:12.
ERROR: /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/noscope/darknet/BUILD:10:1: in srcs attribute of cc_library rule //tensorflow/noscope/darknet:yolo_gpu: file '//tensorflow/noscope/darknet:src/deconvolutional_kernels.cu' is misplaced here (expected .cc, .cpp, .cxx, .c++, .C, .c, .h, .hh, .hpp, .hxx, .inc, .S, .s, .asm, .a, .lib, .pic.a, .lo, .pic.lo, .so, .dylib, .dll, .o or .pic.o). Since this rule was created by the macro 'tf_kernel_library', the error might have been caused by the macro implementation in /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/tensorflow.bzl:634:12.
ERROR: /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/noscope/darknet/BUILD:10:1: in srcs attribute of cc_library rule //tensorflow/noscope/darknet:yolo_gpu: '//tensorflow/noscope/darknet:src/deconvolutional_kernels.cu' does not produce any cc_library srcs files (expected .cc, .cpp, .cxx, .c++, .C, .c, .h, .hh, .hpp, .hxx, .inc, .S, .s, .asm, .a, .lib, .pic.a, .lo, .pic.lo, .so, .dylib, .dll, .o or .pic.o). Since this rule was created by the macro 'tf_kernel_library', the error might have been caused by the macro implementation in /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/tensorflow.bzl:634:12.
ERROR: /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/noscope/darknet/BUILD:10:1: in srcs attribute of cc_library rule //tensorflow/noscope/darknet:yolo_gpu: file '//tensorflow/noscope/darknet:src/dropout_layer_kernels.cu' is misplaced here (expected .cc, .cpp, .cxx, .c++, .C, .c, .h, .hh, .hpp, .hxx, .inc, .S, .s, .asm, .a, .lib, .pic.a, .lo, .pic.lo, .so, .dylib, .dll, .o or .pic.o). Since this rule was created by the macro 'tf_kernel_library', the error might have been caused by the macro implementation in /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/tensorflow.bzl:634:12.
ERROR: /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/noscope/darknet/BUILD:10:1: in srcs attribute of cc_library rule //tensorflow/noscope/darknet:yolo_gpu: '//tensorflow/noscope/darknet:src/dropout_layer_kernels.cu' does not produce any cc_library srcs files (expected .cc, .cpp, .cxx, .c++, .C, .c, .h, .hh, .hpp, .hxx, .inc, .S, .s, .asm, .a, .lib, .pic.a, .lo, .pic.lo, .so, .dylib, .dll, .o or .pic.o). Since this rule was created by the macro 'tf_kernel_library', the error might have been caused by the macro implementation in /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/tensorflow.bzl:634:12.
ERROR: /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/noscope/darknet/BUILD:10:1: in srcs attribute of cc_library rule //tensorflow/noscope/darknet:yolo_gpu: file '//tensorflow/noscope/darknet:src/im2col_kernels.cu' is misplaced here (expected .cc, .cpp, .cxx, .c++, .C, .c, .h, .hh, .hpp, .hxx, .inc, .S, .s, .asm, .a, .lib, .pic.a, .lo, .pic.lo, .so, .dylib, .dll, .o or .pic.o). Since this rule was created by the macro 'tf_kernel_library', the error might have been caused by the macro implementation in /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/tensorflow.bzl:634:12.
ERROR: /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/noscope/darknet/BUILD:10:1: in srcs attribute of cc_library rule //tensorflow/noscope/darknet:yolo_gpu: '//tensorflow/noscope/darknet:src/im2col_kernels.cu' does not produce any cc_library srcs files (expected .cc, .cpp, .cxx, .c++, .C, .c, .h, .hh, .hpp, .hxx, .inc, .S, .s, .asm, .a, .lib, .pic.a, .lo, .pic.lo, .so, .dylib, .dll, .o or .pic.o). Since this rule was created by the macro 'tf_kernel_library', the error might have been caused by the macro implementation in /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/tensorflow.bzl:634:12.
ERROR: /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/noscope/darknet/BUILD:10:1: in srcs attribute of cc_library rule //tensorflow/noscope/darknet:yolo_gpu: file '//tensorflow/noscope/darknet:src/maxpool_layer_kernels.cu' is misplaced here (expected .cc, .cpp, .cxx, .c++, .C, .c, .h, .hh, .hpp, .hxx, .inc, .S, .s, .asm, .a, .lib, .pic.a, .lo, .pic.lo, .so, .dylib, .dll, .o or .pic.o). Since this rule was created by the macro 'tf_kernel_library', the error might have been caused by the macro implementation in /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/tensorflow.bzl:634:12.
ERROR: /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/noscope/darknet/BUILD:10:1: in srcs attribute of cc_library rule //tensorflow/noscope/darknet:yolo_gpu: '//tensorflow/noscope/darknet:src/maxpool_layer_kernels.cu' does not produce any cc_library srcs files (expected .cc, .cpp, .cxx, .c++, .C, .c, .h, .hh, .hpp, .hxx, .inc, .S, .s, .asm, .a, .lib, .pic.a, .lo, .pic.lo, .so, .dylib, .dll, .o or .pic.o). Since this rule was created by the macro 'tf_kernel_library', the error might have been caused by the macro implementation in /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/tensorflow.bzl:634:12.
ERROR: /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/noscope/darknet/BUILD:10:1: in srcs attribute of cc_library rule //tensorflow/noscope/darknet:yolo_gpu: file '//tensorflow/noscope/darknet:src/network_kernels.cu' is misplaced here (expected .cc, .cpp, .cxx, .c++, .C, .c, .h, .hh, .hpp, .hxx, .inc, .S, .s, .asm, .a, .lib, .pic.a, .lo, .pic.lo, .so, .dylib, .dll, .o or .pic.o). Since this rule was created by the macro 'tf_kernel_library', the error might have been caused by the macro implementation in /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/tensorflow.bzl:634:12.
ERROR: /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/noscope/darknet/BUILD:10:1: in srcs attribute of cc_library rule //tensorflow/noscope/darknet:yolo_gpu: '//tensorflow/noscope/darknet:src/network_kernels.cu' does not produce any cc_library srcs files (expected .cc, .cpp, .cxx, .c++, .C, .c, .h, .hh, .hpp, .hxx, .inc, .S, .s, .asm, .a, .lib, .pic.a, .lo, .pic.lo, .so, .dylib, .dll, .o or .pic.o). Since this rule was created by the macro 'tf_kernel_library', the error might have been caused by the macro implementation in /home/zjc/DeepLearning/tensorflow-noscope/tensorflow/tensorflow.bzl:634:12.
ERROR: Analysis of target '//tensorflow/noscope:noscope' failed; build aborted.

can someone help me fix this?

code problem：noscope/Models.py->try_params->nb_classes = params[1]？

First，Thank you for contributing the code.When i read the Models.py,i have a problem that i can't understand.
in noscope/Models.py,the function of try_params(...)，in line 305
nb_classes means the number of the classes,but params[1] is member of list(itertools.product( *[[X_train.shape[1:]], [nb_classes],[32, 64, 128, 256], [32], [0, 1, 2]])),
so params[1]=[(50,50,3),2,32,32,1]
why nb_classes = params[1]？I think nb_classes=params[1][2],because params[1][2] is [nb_classes].
am I wrong？if i am wrong，please help me understand it，thank you！

No such file or directory:train_918000_1836000.csv

Firstly, thanks a lot for updating your tutorial in time , and i follow your steps which you required.
Everything seems good until the error appears:

and in that directory i only to find as following:

it generate the train_918000_1836000.log but no train_918000_1836000.csv
I do not know where the problem is,can you help me?
(PS:for the mp4 is too large, i just cut it to 3000 frames and i also changge the NUM_FRAMES="3000")

How to understand YOLO_LABELS in noscope_motherdog.py

I see this in noscope_motherdog.py:

YOLO_LABELS["coral-reef-long"] = (
0,
[("coral-reef-long_convnet_32_32_1.pb", None),
("coral-reef-long_convnet_32_32_2.pb", None),],
648000 + 1188000,
1188000,
648000 + 1188000 * 2,
1188000
)

So what do those numbers mean? Is there a document explaining those?
I really need to understand those so I can add my own
YOLO_LABELS["my-video"] session with the correct numbers to have Yolo process my video.

Right now, Yolo is not processing my video correctly because those numbers are not setup right.