thtrieu / darkflow

Translate darknet to tensorflow. Load trained weights, retrain/fine-tune using tensorflow, export constant graph def to mobile devices

Hi there 👋

darkflow's Issues

using graph-yolo.pb error android example

i have generate two file graph-tiny-yolo-voc.pb and graph-yolo.pb.

the file "graph-tiny-yolo-voc.pb" is ok, but graph-yolo.pb error

I/native: tensorflow_inference_jni.cc:139 Creating TensorFlow graph from GraphDef.
E/native: tensorflow_inference_jni.cc:146 Could not create TensorFlow graph: Invalid argument: No OpKernel was registered to support Op 'ExtractImagePatches' with these attrs. Registered devices: [CPU], Registered kernels:

[[Node: ExtractImagePatches = ExtractImagePatchesT=DT_FLOAT, ksizes=[1, 2, 2, 1], padding="VALID", rates=[1, 1, 1, 1], strides=[1, 2, 2, 1]]]
E/tensorflow: TensorFlowYoloDetector: TF init status: 3

How to receive .pb file with the least efforts?

Thanks for repository! Sorry, I don't understand anything from the documentation.
So, I have an iOS tensorflow app, it works with tensorflow_inception_graph.pb. I'd like to get YOLO model in such format. For now, I don't care if it's YOLO-small, or YOLO (or tiny).

What do all these non-existing filenames mean?

# 1. Load yolo-tiny.weights
./flow --model cfg/yolo-tiny.cfg --load bin/yolo-tiny.weights

There isn't yolo-tiny.cfg anywhere.

What I've found on YOLO website is tiny-yolo-voc and yolo-voc.

When trying yolo-voc the model isn't loading:

Mac-mini-Olya:darkflow Olya$ ./flow --model cfg/yolo-voc.cfg --load bin/yolo-voc.weights
Parsing ./cfg/yolo-voc.cfg
Traceback (most recent call last):
  File "./flow", line 42, in <module>
    tfnet = TFNet(FLAGS)
  File "/Users/Olya/Documents/want_protobuf/darkflow/net/build.py", line 34, in __init__
    darknet = Darknet(FLAGS)
  File "/Users/Olya/Documents/want_protobuf/darkflow/dark/darknet.py", line 17, in __init__
    src_parsed = self.parse_cfg(self.src_cfg, FLAGS)
  File "/Users/Olya/Documents/want_protobuf/darkflow/dark/darknet.py", line 68, in parse_cfg
    for i, info in enumerate(cfg_layers):
  File "/Users/Olya/Documents/want_protobuf/darkflow/cfg/process.py", line 294, in cfg_yielder
    lx = layers[x]; xtype = lx['type']
TypeError: list indices must be integers, not unicode

When trying tiny-yolo-voc it's loading (./flow --model cfg/tiny-yolo-voc.cfg --load bin/tiny-yolo-voc.weights).
But the training doesn't work:
both
./flow --train --model cfg/tiny-yolo.cfg --load -1
and
./flow --model cfg/tiny-yolo.cfg --train --trainer adam
give an error AssertionError: Loss type [region] not implemented.

So, my questions are:

1. Can .pb file with yolo-to-tensorflow model be received only after training process (because of ckpt files) or is there another way?
2. Which .cfg file should be used for training?

Ideally, I'd like to see real 3 commands (load, train, save) with real filenames (urls), not some imaginary stuff. :)

Out of Memory

Hi @thtrieu,

What the minimal requirement for the GPU device.
I use GTX 1080 and even set the batch = 1 and got out of memory?

darkflow$ ./flow --model cfg/tiny-yolo-voc.cfg --load bin/tiny-yolo-voc.weights --train --gpu 1.0
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcurand.so locally
Parsing ./cfg/tiny-yolo-voc.cfg
Parsing cfg/tiny-yolo-voc.cfg
Loading bin/tiny-yolo-voc.weights ...
Successfully identified 63471556 bytes
Finished in 0.00466394424438s

Building net ...
Source | Train? | Layer description | Output size
-------+--------+----------------------------------+---------------
| | input | (?, 96, 96, 3)
Load | Yep! | conv 3x3p1_1 +bnorm leaky | (?, 96, 96, 16)
Load | Yep! | maxp 2x2p0_2 | (?, 48, 48, 16)
Load | Yep! | conv 3x3p1_1 +bnorm leaky | (?, 48, 48, 32)
Load | Yep! | maxp 2x2p0_2 | (?, 24, 24, 32)
Load | Yep! | conv 3x3p1_1 +bnorm leaky | (?, 24, 24, 64)
Load | Yep! | maxp 2x2p0_2 | (?, 12, 12, 64)
Load | Yep! | conv 3x3p1_1 +bnorm leaky | (?, 12, 12, 128)
Load | Yep! | maxp 2x2p0_2 | (?, 6, 6, 128)
Load | Yep! | conv 3x3p1_1 +bnorm leaky | (?, 6, 6, 256)
Load | Yep! | maxp 2x2p0_2 | (?, 3, 3, 256)
Load | Yep! | conv 3x3p1_1 +bnorm leaky | (?, 3, 3, 512)
Load | Yep! | maxp 2x2p0_1 | (?, 3, 3, 512)
Load | Yep! | conv 3x3p1_1 +bnorm leaky | (?, 3, 3, 1024)
Load | Yep! | conv 3x3p1_1 +bnorm leaky | (?, 3, 3, 1024)
Load | Yep! | conv 1x1p0_1 linear | (?, 3, 3, 125)
-------+--------+----------------------------------+---------------
GPU mode with 1.0 usage
cfg/tiny-yolo-voc.cfg loss hyper-parameters:
H = 3
W = 3
box = 5
classes = 5
scales = [1.0, 5.0, 1.0, 1.0]
Building cfg/tiny-yolo-voc.cfg loss
Building cfg/tiny-yolo-voc.cfg train op
I tensorflow/core/common_runtime/gpu/gpu_device.cc:885] Found device 0 with properties:
name: GeForce GTX 1080
major: 6 minor: 1 memoryClockRate (GHz) 1.86
pciBusID 0000:03:00.0
Total memory: 7.92GiB
Free memory: 7.66GiB
I tensorflow/core/common_runtime/gpu/gpu_device.cc:906] DMA: 0
I tensorflow/core/common_runtime/gpu/gpu_device.cc:916] 0: Y
I tensorflow/core/common_runtime/gpu/gpu_device.cc:975] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1080, pci bus id: 0000:03:00.0)
E tensorflow/stream_executor/cuda/cuda_driver.cc:1002] failed to allocate 7.92G (8505458688 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
Segmentation fault (core dumped)

How to fine-tune?

Firstly, let me say thanks and congratulations on this ambitious and impressive project.

I've been perusing the code-base and I don't understand it very well. I would like to fine-tune YOLO to my own dataset, using a checkpoint that was learnt on VOC/COCO data.

Just as a test-run, to see what would happen, I ran

./flow --train --model cfg/tiny-yolo.cfg --load bin/tiny-yolo.weights

which came back with the error:

...
Load  |  Yep!  | conv 3x3p1_1  +bnorm  leaky      | (?, 13, 13, 1024)
Load  |  Yep!  | conv 3x3p1_1  +bnorm  leaky      | (?, 13, 13, 1024)
Load  |  Yep!  | conv 1x1p0_1    linear           | (?, 13, 13, 425)
-------+--------+----------------------------------+---------------
Running entirely on CPU
Traceback (most recent call last):
 File "./flow", line 42, in <module>
   tfnet = TFNet(FLAGS)
 File "/home/hal9000/Sources/darkflow/net/build.py", line 51, in __init__
   self.setup_meta_ops()
 File "/home/hal9000/Sources/darkflow/net/build.py", line 94, in setup_meta_ops
   if self.FLAGS.train: self.build_train_op()
 File "/home/hal9000/Sources/darkflow/net/help.py", line 15, in build_train_op
   self.framework.loss(self.out)
 File "/home/hal9000/Sources/darkflow/net/vanilla/train.py", line 9, in loss
   'Loss type {} not implemented'.format(loss_type)
AssertionError: Loss type [region] not implemented

So I ask:

(1) Which options on the flow shell script do I need to run fine-tuning?
(2) How do I point to my dataset, and how should it be formatted?

Thanks again.

Low fps on tiny memory devices

Thanks for your contribution to build a TensorFlow version of Darknet.
I'm trying to run darkflow on NVIDIA Jetson TX1, which has 4GB memory and so-called 1TFlops GPU. At first time I run the webcam demo of tiny-yolo (version 1), it calls MEMORY OUT.

# ubuntu @ tegra-ubuntu in ~/Github/darkflow on git:master x [17:29:21]
$ ./flow --demo --model ~/tiny-yolo.train.cfg --load ~/tiny-yolo_201611160227.weights --gpu 1.0
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcurand.so locally
Parsing /home/ubuntu/tiny-yolo.train.cfg
Loading /home/ubuntu/tiny-yolo_201611160227.weights ...
Successfully identified 113568356 bytes
Finished in 0.0166881084442s

Building net ...
Source | Train? | Layer description                | Output size
-------+--------+----------------------------------+---------------
       |        | input                            | (?, 448, 448, 3)
 Load  |  Yep!  | conv 3x3p1_1  +bnorm  leaky      | (?, 448, 448, 16)
 Load  |  Yep!  | maxp 2x2p0_2                     | (?, 224, 224, 16)
 Load  |  Yep!  | conv 3x3p1_1  +bnorm  leaky      | (?, 224, 224, 32)
 Load  |  Yep!  | maxp 2x2p0_2                     | (?, 112, 112, 32)
 Load  |  Yep!  | conv 3x3p1_1  +bnorm  leaky      | (?, 112, 112, 64)
 Load  |  Yep!  | maxp 2x2p0_2                     | (?, 56, 56, 64)
 Load  |  Yep!  | conv 3x3p1_1  +bnorm  leaky      | (?, 56, 56, 128)
 Load  |  Yep!  | maxp 2x2p0_2                     | (?, 28, 28, 128)
 Load  |  Yep!  | conv 3x3p1_1  +bnorm  leaky      | (?, 28, 28, 256)
 Load  |  Yep!  | maxp 2x2p0_2                     | (?, 14, 14, 256)
 Load  |  Yep!  | conv 3x3p1_1  +bnorm  leaky      | (?, 14, 14, 512)
 Load  |  Yep!  | maxp 2x2p0_2                     | (?, 7, 7, 512)
 Load  |  Yep!  | conv 3x3p1_1  +bnorm  leaky      | (?, 7, 7, 1024)
 Load  |  Yep!  | conv 3x3p1_1  +bnorm  leaky      | (?, 7, 7, 256)
 Load  |  Yep!  | flat                             | (?, 12544)
 Load  |  Yep!  | full 12544 x 1573  linear        | (?, 1573)
-------+--------+----------------------------------+---------------
GPU mode with 1.0 usage
I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:892] ARM has no NUMA node, hardcoding to return zero
I tensorflow/core/common_runtime/gpu/gpu_device.cc:951] Found device 0 with properties:
name: NVIDIA Tegra X1
major: 5 minor: 3 memoryClockRate (GHz) 0.072
pciBusID 0000:00:00.0
Total memory: 3.90GiB
Free memory: 1.42GiB
I tensorflow/core/common_runtime/gpu/gpu_device.cc:972] DMA: 0
I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] 0:   Y
I tensorflow/core/common_runtime/gpu/gpu_device.cc:1041] Creating TensorFlow device (/gpu:0) -> (device: 0, name: NVIDIA Tegra X1, pci bus id: 0000:00:00.0)
E tensorflow/stream_executor/cuda/cuda_driver.cc:965] failed to allocate 3.90G (4188778496 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
E tensorflow/stream_executor/cuda/cuda_driver.cc:965] failed to allocate 3.51G (3769900544 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
E tensorflow/stream_executor/cuda/cuda_driver.cc:965] failed to allocate 3.16G (3392910336 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
E tensorflow/stream_executor/cuda/cuda_driver.cc:965] failed to allocate 2.84G (3053619200 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
E tensorflow/stream_executor/cuda/cuda_driver.cc:965] failed to allocate 2.56G (2748257280 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
E tensorflow/stream_executor/cuda/cuda_driver.cc:965] failed to allocate 2.30G (2473431552 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
E tensorflow/stream_executor/cuda/cuda_driver.cc:965] failed to allocate 2.07G (2226088448 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc
[1]    2527 abort      ./flow --demo --model ~/tiny-yolo.train.cfg --load  --gpu 1.0

So I add some swap space to make it run successfully. It do runs, but only get 4fps. However, it can get up to 12fps on original darknet framework with the same model and weights. And it only take 1GB memory to run.

# ubuntu @ tegra-ubuntu in ~/Github/darkflow on git:master x [17:29:57] C:134
$ ./flow --demo --model ~/tiny-yolo.train.cfg --load ~/tiny-yolo_201611160227.weights --gpu 1.0
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcurand.so locally
Parsing /home/ubuntu/tiny-yolo.train.cfg
Loading /home/ubuntu/tiny-yolo_201611160227.weights ...
Successfully identified 113568356 bytes
Finished in 0.0139830112457s

Building net ...
Source | Train? | Layer description                | Output size
-------+--------+----------------------------------+---------------
       |        | input                            | (?, 448, 448, 3)
 Load  |  Yep!  | conv 3x3p1_1  +bnorm  leaky      | (?, 448, 448, 16)
 Load  |  Yep!  | maxp 2x2p0_2                     | (?, 224, 224, 16)
 Load  |  Yep!  | conv 3x3p1_1  +bnorm  leaky      | (?, 224, 224, 32)
 Load  |  Yep!  | maxp 2x2p0_2                     | (?, 112, 112, 32)
 Load  |  Yep!  | conv 3x3p1_1  +bnorm  leaky      | (?, 112, 112, 64)
 Load  |  Yep!  | maxp 2x2p0_2                     | (?, 56, 56, 64)
 Load  |  Yep!  | conv 3x3p1_1  +bnorm  leaky      | (?, 56, 56, 128)
 Load  |  Yep!  | maxp 2x2p0_2                     | (?, 28, 28, 128)
 Load  |  Yep!  | conv 3x3p1_1  +bnorm  leaky      | (?, 28, 28, 256)
 Load  |  Yep!  | maxp 2x2p0_2                     | (?, 14, 14, 256)
 Load  |  Yep!  | conv 3x3p1_1  +bnorm  leaky      | (?, 14, 14, 512)
 Load  |  Yep!  | maxp 2x2p0_2                     | (?, 7, 7, 512)
 Load  |  Yep!  | conv 3x3p1_1  +bnorm  leaky      | (?, 7, 7, 1024)
 Load  |  Yep!  | conv 3x3p1_1  +bnorm  leaky      | (?, 7, 7, 256)
 Load  |  Yep!  | flat                             | (?, 12544)
 Load  |  Yep!  | full 12544 x 1573  linear        | (?, 1573)
-------+--------+----------------------------------+---------------
GPU mode with 1.0 usage
I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:892] ARM has no NUMA node, hardcoding to return zero
I tensorflow/core/common_runtime/gpu/gpu_device.cc:951] Found device 0 with properties:
name: NVIDIA Tegra X1
major: 5 minor: 3 memoryClockRate (GHz) 0.072
pciBusID 0000:00:00.0
Total memory: 3.90GiB
Free memory: 1.31GiB
I tensorflow/core/common_runtime/gpu/gpu_device.cc:972] DMA: 0
I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] 0:   Y
I tensorflow/core/common_runtime/gpu/gpu_device.cc:1041] Creating TensorFlow device (/gpu:0) -> (device: 0, name: NVIDIA Tegra X1, pci bus id: 0000:00:00.0)
E tensorflow/stream_executor/cuda/cuda_driver.cc:965] failed to allocate 3.90G (4188778496 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
E tensorflow/stream_executor/cuda/cuda_driver.cc:965] failed to allocate 3.51G (3769900544 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
E tensorflow/stream_executor/cuda/cuda_driver.cc:965] failed to allocate 3.16G (3392910336 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
E tensorflow/stream_executor/cuda/cuda_driver.cc:965] failed to allocate 2.84G (3053619200 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
E tensorflow/stream_executor/cuda/cuda_driver.cc:965] failed to allocate 2.56G (2748257280 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
E tensorflow/stream_executor/cuda/cuda_driver.cc:965] failed to allocate 2.30G (2473431552 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
E tensorflow/stream_executor/cuda/cuda_driver.cc:965] failed to allocate 2.07G (2226088448 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
Finished in 56.6668601036s

HIGHGUI ERROR: V4L/V4L2: VIDIOC_S_CROP
Press [ESC] to quit camera demo
E tensorflow/stream_executor/cuda/cuda_driver.cc:965] failed to allocate 7.80G (8377556992 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
W tensorflow/core/common_runtime/bfc_allocator.cc:213] Ran out of memory trying to allocate 2.15GiB. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory is available.
2.720 FPS

I've tried to modify batchsize and subdivisions but it still run out of memory.

Are there any methods to reduce the memory usage?

Crash after install

Hi -
I have some experience using Darknet/YOLO, and was excited to learn that you have reimplemented it using TensorFlow. I have TensorFlow 0.12rc1 installed, and Python2.7. TF examples work.

I cloned your repository, and grabbed cfg and weights for tiny-yolo. When I attempted to run it using your instructions, however, I get an error:

rob@skynet1:~/darkflow$ ./flow --model yolo-voc.cfg --load yolo-voc.weights
I tensorflow/stream_executor/dso_loader.cc:120] successfully opened CUDA library libcublas.so.8.0 locally
I tensorflow/stream_executor/dso_loader.cc:120] successfully opened CUDA library libcudnn.so.5 locally
I tensorflow/stream_executor/dso_loader.cc:120] successfully opened CUDA library libcufft.so.8.0 locally
I tensorflow/stream_executor/dso_loader.cc:120] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:120] successfully opened CUDA library libcurand.so.8.0 locally
Parsing yolo-voc.cfg
Loading yolo-voc.weights ...
Successfully identified 268632452 bytes
Finished in 0.0848739147186s

Building net ...
Source | Train? | Layer description | Output size
-------+--------+----------------------------------+---------------
Traceback (most recent call last):
File "./flow", line 42, in
tfnet = TFNet(FLAGS)
File "/home/rob/darkflow/net/build.py", line 49, in init
self.build_forward()
File "/home/rob/darkflow/net/build.py", line 69, in build_forward
state = op_create(*args)
File "/home/rob/darkflow/net/ops/init.py", line 27, in op_create
return op_typeslayer_type
File "/home/rob/darkflow/net/ops/baseop.py", line 42, in init
self.forward()
File "/home/rob/darkflow/net/ops/convolution.py", line 66, in forward
temp = self.batchnorm(self.lay, temp)
File "/home/rob/darkflow/net/ops/convolution.py", line 78, in batchnorm
is_training = layer.h['is_training'])
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/framework/python/ops/arg_scope.py", line 177, in func_with_args
return func(*args, **current_args)
TypeError: batch_norm() got an unexpected keyword argument 'initializers'
rob@skynet1:~/darkflow$

I am trying to determine if this is a TF bug or otherwise. Any clues?
Thanks,
Rob

batchnorm() unexpected keyword argument

First, thanks a lot for your work! I'm trying to get it to run but fail at the second step.

./flow --h does what it's intended to do but when I move on to
./flow --model cfg/yolo-tiny.cfg --load bin/yolo-tiny.weights then I get the following error:

batch_norm() got an unexpected keyword argument 'initializers'

Which relates to the function batchnorm in convolution.py:

def batchnorm(self, layer, inp):
	if not self.var:
		temp = (inp - layer.w['moving_mean'])
		temp /= (np.sqrt(layer.w['moving_variance']) + 1e-5)
		temp *= layer.w['gamma']
		return temp
	else: return slim.batch_norm(inp, 
		center = False, scale = True, epsilon = 1e-5,
		initializers = layer.w, scope = self.scope,
		is_training = layer.h['is_training'])

And more precisely initializers = layer.w

After looking around, for instance here, it seems that slim.batch_norm() doesn't use an initializers argument. Is there anything I'm missing? Should I install a specific version of tensorflow to run this?

Tensorflow 1.0 support

When I upgraded tensorflow from 0.12.1 to 1.0.0, I got the error:

`
Traceback (most recent call last):
File "./flow", line 42, in
tfnet = TFNet(FLAGS)
File "/home/chenxi/workspace/facedet/darkflow/net/build.py", line 50, in init
self.build_forward()
File "/home/chenxi/workspace/facedet/darkflow/net/build.py", line 70, in build_forward
state = op_create(*args)
File "/home/chenxi/workspace/facedet/darkflow/net/ops/init.py", line 27, in op_create
return op_typeslayer_type
File "/home/chenxi/workspace/facedet/darkflow/net/ops/baseop.py", line 42, in init
self.forward()
File "/home/chenxi/workspace/facedet/darkflow/net/ops/convolution.py", line 72, in forward
temp = self.batchnorm(self.lay, temp)
File "/home/chenxi/workspace/facedet/darkflow/net/ops/convolution.py", line 92, in batchnorm
return slim.batch_norm(inp, **args)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/contrib/framework/python/ops/arg_scope.py", line 177, in func_with_args
return func(*args, **current_args)

TypeError: batch_norm() got an unexpected keyword argument 'initializers'
`

But when I switched the version back to 0.12.1, it worked fine.
Could you please update your tensorflow and fix it?
Thank you for your excellent work!

interpreting decode()

Thanks for sharing this awesome project :)

I'd like to fork this repo for a research project, to experiment with yolo, using alternative input types (multiple different spectral range images, e.g.).

I tried a tf implementation myself, though I got stuck on the error definition, regarding the bounding boxes error definition..
It seems you solved this problem, using numpy operations, though I don't quite follow your steps.
As far as I understand you perform these operations in decode(), tfnet.py, r140.

I wondered whether you could add come comments in decode(), explaining your solution.
Some statements as r169 are a bit magical to me :)
I'd like to fully understand your steps, so I could modify your code if necessary.

May the force be with you,

Marc

question about initializing model and reusing layers - error

I am following tutorial with the goal to retrain the model on my own classes. Thanks for such awesome work!
I encounter some issues. Let me ask if what I do is correct and then why it fails:
I download any desired pair of cfg+weights from http://pjreddie.com/darknet/yolo/

Following the instructions I try to execute and get respectively errors as below:

# 2. To completely initialize a model, leave the --load option
./flow --model cfg/tiny-yolo.cfg 

Parsing cfg/tiny-yolo.cfg
Traceback (most recent call last):
  File "./flow", line 42, in <module>
    tfnet = TFNet(FLAGS)
  File "/home/john/workspace/darkflow/net/build.py", line 34, in __init__
    darknet = Darknet(FLAGS)
  File "/home/john/workspace/darkflow/dark/darknet.py", line 17, in __init__
    src_parsed = self.parse_cfg(self.src_cfg, FLAGS)
  File "/home/john/workspace/darkflow/dark/darknet.py", line 68, in parse_cfg
    for i, info in enumerate(cfg_layers):
  File "/home/john/workspace/darkflow/cfg/process.py", line 66, in cfg_yielder
    layers, meta = parser(model); yield meta;
  File "/home/john/workspace/darkflow/cfg/process.py", line 20, in parser
    lines = [line.decode() for line in lines]	
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 41: ordinal not in range(128)

# 3. It is useful to reuse the first identical layers of tiny for 3c
./flow --test test/ --model cfg/yolo-tiny.cfg --load bin/yolo-tiny.weights

/home/john/workspace/darkflow/dark/darknet.py:54: UserWarning: ./cfg/yolo-tiny.cfg not found, use cfg/yolo-tiny.cfg instead
  cfg_path, FLAGS.model))
Parsing cfg/yolo-tiny.cfg
Traceback (most recent call last):
  File "./flow", line 42, in <module>
    tfnet = TFNet(FLAGS)
  File "/home/john/workspace/darkflow/net/build.py", line 34, in __init__
    darknet = Darknet(FLAGS)
  File "/home/john/workspace/darkflow/dark/darknet.py", line 17, in __init__
    src_parsed = self.parse_cfg(self.src_cfg, FLAGS)
  File "/home/john/workspace/darkflow/dark/darknet.py", line 68, in parse_cfg
    for i, info in enumerate(cfg_layers):
  File "/home/john/workspace/darkflow/cfg/process.py", line 66, in cfg_yielder
    layers, meta = parser(model); yield meta;
  File "/home/john/workspace/darkflow/cfg/process.py", line 17, in parser
    with open(model, 'rb') as f:
IOError: [Errno 2] No such file or directory: 'cfg/yolo-tiny.cfg'

Please consider that I downloaded the yolo-tiny.cfg from darknet repository directly into /cfg, it is there.
Do I miss something? Maybe I don't fully understand concept of reusing weights with the chosen models. But choosing the same pair I supposed would work. Thanks!

Result is different between darknet and darkflow

Darknet

Run

$./darknet detector test -gpus 0,1 cfg/voc.data cfg/tiny-yolo-voc.cfg /data/dark_backup/tiny-yolo-voc_final.weights data/person.jpg

Inference result from darknet

Darkflow

Run

$./flow --test test/ --model ~/workspace/darknet/cfg/tiny-yolo-voc.cfg --load /data/dark_backup/tiny-yolo-voc_final.weights --gpu 1.0

Inference result rom darkflow

I wonder why these two images show different output. Is there anything I missed?

thanks in advance!

ValueError: cannot convert float NaN to integer

Dear
Thank you very much for posting Yolo in tensorflow.
I try the demo and get the following error. Could you please have a look?
First, I download the yolotiny.weights from Yolo website.

Then,

python clean.py /home/karl/Documents/VOCdevkit/VOC2012/Annotations
[===================>]100%
Statistics:
pottedplant: 13442
person: 17401
tvmonitor: 15512
Dataset size: 26089

At last, I run the test code:

python tensor.py --test data --model tiny
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcurand.so locally
parsing yolo-tiny.cfg
Traceback (most recent call last):
  File "tensor.py", line 37, in <module>
    yoloNet = YOLO(FLAGS.model + int(step > 0) * '-{}'.format(step))
  File "/home/karl/Documents/online_code/yolotf/Yolo.py", line 57, in __init__
    self.build(model)
  File "/home/karl/Documents/online_code/yolotf/Yolo.py", line 71, in build
    for i, info in enumerate(layers):
  File "/home/karl/Documents/online_code/yolotf/configs/process.py", line 60, in cfg_yielder
    size = int(size)
ValueError: cannot convert float NaN to integer

How to fix that?
Thank you very much.

Random scale/translate send object(s) out of bound

while trainig on my data
I got this error
Random scale/translate send object(s) out of bound
as shown in the image
and then it starts to test on the testing data and store the results to results/ folder

I think the model doesn't do the training phase properly
and the problem is in the shuffle function in TFnet.py file
in the try statement it has some error and goes to the except
any I deas about this error?

OSError: [Errno 2] No such file or directory: '../pascal/VOCdevkit/ANN'

When I run python clean.py I get this error : OSError: [Errno 2] No such file or directory: '../pascal/VOCdevkit/ANN'

I assume the ANN folder might've been already downloaded with the repo , but couldnt find it

Guidelines on training for PascalVOC data?

Hi -
I can successfully run darkflow in demo mode with the camera. Great!
I am trying to start a training run with PascalVOC dataset, and having trouble.
I have the VOC2012 set downloaded.
Question: I need to create the labels files from the VOC Annotations folder, does darkflow do that automatically with the --annotation <path_to_VOC_Annotations> flag as part of the training, or do I call it before running with the --train flag?

Any guidelines on setup here would be appreciated.
Thanks,
Rob

Using pretrained convolutional weights

The Yolo training examples use convolutional weights pretrained on ImageNet http://pjreddie.com/darknet/yolov1/ . These seem to be created using the Darknet partial command which saves only the weights of the first N network layers (e.g. 24).

I wonder whether there is a way to use pretrained CNN weights with flow in a similar fashion?

Should I train the whole network from scratch? What is the down side?

I tried with the --load option like this:

./flow --train --model mymodel --load ../darknet/darknet.conv.weights --trainer adam --gpu 0.8

flow does not seem to behave as I expected though. It loads configs/mymodel.cfg but then it also tries to load configs/darknet.conf.cfg .

I tried to cheat and rename darknet.conf.weights to yolo-tiny.weights so that yolo-tiny.cfg is picked up but I end up with an error message like this (makes sense):

Error: Configuration suggests a bigger size than ./bin/yolo-tiny.weights actually is.

I wonder whether I can save the pretrained Darknet weights in a checkpoint somehow?...

It seems that #5 is related, but not the same use case.

Incompatible shapes

Hi @thtrieu,

I got the following error:
InvalidArgumentError (see above for traceback): Incompatible shapes: [40,196,5,2] vs. [16,196,5,2]
[[Node: Minimum = Minimum[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"](add_2, _recv_Placeholder_6_0)]]

I use this command: ./flow --model cfg/tiny-yolo-voc.cfg --load bin/tiny-yolo-voc.weights --train

Use pretrained with custom output class

Hi,

When I was training the new model using tiny-yolo-voc with custom dataset with five number of class, and also using tiny-yolo-voc.weights, I got the following error:

Loading bin/tiny-yolo-voc.weights ...
Traceback (most recent call last):
File "./flow", line 42, in
tfnet = TFNet(FLAGS)
File "darkflow/net/build.py", line 34, in init
darknet = Darknet(FLAGS)
File "darkflow/dark/darknet.py", line 27, in init
self.load_weights()
File "darkflow/dark/darknet.py", line 82, in load_weights
wgts_loader = loader.create_loader(*args)
File "darkflow/utils/loader.py", line 104, in create_loader
return load_type(path, cfg)
File "darkflow/utils/loader.py", line 18, in init
self.load(*args)
File "darkflow/utils/loader.py", line 76, in load
walker.offset, walker.size)
AssertionError: expect 63164056 bytes, found 63471556

ValueError: invalid shape in fixed-type tuple: dtype size in bytes must fit into a C int.

When I try to use yolo.cfg with the 753 MB yolov1.weights (renamed to bin/yolo.weights) from http://pjreddie.com/darknet/yolov1/, I get the below numpy error:

$ python flow --testset datas/ --model yolo --threshold 0.1 --gpu 0.8
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcurand.so locally
Parsing ./configs/yolo.cfg
Last convolutional kernel size = 26.5098364494
Loading ./bin/yolo.weights ...
Traceback (most recent call last):
  File "flow", line 31, in <module>
    darknet = Darknet(FLAGS)
  File "/home/ktenchov/work/yolotf/darknet/darknet.py", line 36, in __init__
    self.load_weights(weight_file)
  File "/home/ktenchov/work/yolotf/darknet/darknet.py", line 61, in load_weights
    l.load(loader)
  File "/home/ktenchov/work/yolotf/darknet/darkop.py", line 76, in load
    weight_array = loader(weight_size)
  File "/home/ktenchov/work/yolotf/darknet/darkop.py", line 16, in __call__
    dtype = '({})float32,'.format(length))
  File "/usr/local/lib/python2.7/dist-packages/numpy/core/memmap.py", line 225, in __new__
    descr = dtypedescr(dtype)
ValueError: invalid shape in fixed-type tuple: dtype size in bytes must fit into a C int.

Using yolo-tiny.cfg with the 103 MB tiny-yolo.weights (renamed to bin/yolo-tiny.cfg)

I wonder whether I am missing something obvious?

I am using darktf revision c963245 on a 64-bit Ubuntu 14.04 LTS machine. Here are the relevant Python dependencies:

$ pip list
numpy (1.11.2)
pip (1.5.4)
scipy (0.13.3)
tensorflow (0.11.0)

data augmentation

Hi,

How to activate data augmentation in Yolo V2 ?

weights for models

I'm confused, I'm sure this is a silly question. Where are the .weights files?
specifically I'm looking for the coco models

Thanks!

question about the implemenetation of darkflow

I'm trying to learn darknet and this project is fantastic! Thanks for sharing it.
I have a few questions about the implementation details for the darkflow.

1. What are the design considerations for the classes net.build.TFNet and dark.darknet.Darknet? It seems to me that dark.darknet.Darknet is used to hold the structure and data from cfg and weights files, and net.build.TFNet builds the tensorflow network from dark.darknet.Darknet. Is this understanding correct?
2. Are the net.yolo and net.yolov2 classes share the same operators and preprocessing processes? What are the differences between the two versions of tiny-yolo (darkflow/cfg/tiny-yolo.cfg VS darkflow/cfg/v1/tiny-yolo.cfg) in terms of the network structure and implementation in darkflow?

Thanks.

Guideline to train against other datasets with different classes

A guideline to train against other datasets such as the udacity self driving dataset would be much appreciated.

Do I create a labels.txt in the root folder, and specify a model name outside of coco_models and voc_models listed in darkflow/misc.py?

how to extract and save all layers weights except the last detection layer?

process.py decode lines as unicode and causes exceptions using python2.7

Hi,

I was using python2.7 and trying to run the flow program on yolo v2 (cfg/yolo.cfg and bin/yolo.weights). However, I always get the following exception:

Parsing ./cfg/yolo.cfg
Traceback (most recent call last):
  File "./flow", line 42, in <module>
    tfnet = TFNet(FLAGS)
  File "/home/tammy/SOURCES/darkflow/net/build.py", line 34, in __init__
    darknet = Darknet(FLAGS)
  File "/home/tammy/SOURCES/darkflow/dark/darknet.py", line 17, in __init__
    src_parsed = self.parse_cfg(self.src_cfg, FLAGS)
  File "/home/tammy/SOURCES/darkflow/dark/darknet.py", line 68, in parse_cfg
    for i, info in enumerate(cfg_layers):
  File "/home/tammy/SOURCES/darkflow/cfg/process.py", line 294, in cfg_yielder
    lx = layers[x]; xtype = lx['type']
TypeError: list indices must be integers, not unicode

Then, I realize it was because

[route]
layers=-1,-3

in the yolo.cfg causes the problem.

In line 20, process.py decode lines as unicode. However, it checks if type(routes) is str instead of unicode.

I fixed it in my branch by trying to convert routes into integer and handle the exception if the line includes more than one number separated by comma.

error when run "./flow --h"

I am trying to use ./flow --h and I got this error

Traceback (most recent call last):
File "./flow", line 3, in
from net.build import TFNet
File "/home/jiangyug09/jiangyuguang/darkflow-master/net/build.py", line 10, in
class TFNet(object):
File "/home/jiangyug09/jiangyuguang/darkflow-master/net/build.py", line 16, in TFNet
'adagradDA': tf.train.AdagradDAOptimizer,
AttributeError: 'module' object has no attribute 'AdagradDAOptimizer'

How to generate .pb file of yolo.cfg

Hi Thtrieu,
I meet the issue when I using darkflow to generate pb file of yolo.cfg

$ ./flow --model cfg/yolo.cfg --load ../darknet/yolo-voc_final.weights --savepb

and i get the following error:

I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcurand.so locally
/home/hankwu/DeepLearning/darkflow/dark/darknet.py:54: UserWarning: ./cfg/yolo-voc_final.cfg not found, use cfg/yolo.cfg instead
cfg_path, FLAGS.model))
Parsing cfg/yolo.cfg
Traceback (most recent call last):
File "./flow", line 42, in
tfnet = TFNet(FLAGS)
File "/home/hankwu/DeepLearning/darkflow/net/build.py", line 34, in init
darknet = Darknet(FLAGS)
File "/home/hankwu/DeepLearning/darkflow/dark/darknet.py", line 17, in init
src_parsed = self.parse_cfg(self.src_cfg, FLAGS)
File "/home/hankwu/DeepLearning/darkflow/dark/darknet.py", line 68, in parse_cfg
for i, info in enumerate(cfg_layers):
File "/home/hankwu/DeepLearning/darkflow/cfg/process.py", line 294, in cfg_yielder
lx = layers[x]; xtype = lx['type']
TypeError: list indices must be integers, not unicode

How can I fix it :)

Thank you :)

cuDevicePrimaryCtxSetFlags not found

Hi all,
i tried running flow with following argument
python flow --model cfg/yolo.cfg --load bin/yolo.weights --demo camera --gpu 1.0

got below output

I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\dso_loader.cc:135] successfully opened CUDA library cublas64_80.dll locally
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\dso_loader.cc:135] successfully opened CUDA library cudnn64_5.dll locally
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\dso_loader.cc:135] successfully opened CUDA library cufft64_80.dll locally
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\dso_loader.cc:135] successfully opened CUDA library nvcuda.dll locally
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\dso_loader.cc:135] successfully opened CUDA library curand64_80.dll locally
Parsing ./cfg/yolo.cfg
Parsing cfg/yolo.cfg
Loading bin/yolo.weights ...
Successfully identified 269862452 bytes
Finished in 0.020000934600830078s

Building net ...
E c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "BestSplits" device_type: "CPU"') for unknown op: Bes
E c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "CountExtremelyRandomStats" device_type: "CPU"') for
elyRandomStats
E c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "FinishedNodes" device_type: "CPU"') for unknown op:
E c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "GrowTree" device_type: "CPU"') for unknown op: GrowT
E c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "ReinterpretStringToFloat" device_type: "CPU"') for u
tringToFloat
E c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "SampleInputs" device_type: "CPU"') for unknown op: S
E c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "ScatterAddNdim" device_type: "CPU"') for unknown op:
E c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "TopNInsert" device_type: "CPU"') for unknown op: Top
E c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "TopNRemove" device_type: "CPU"') for unknown op: Top
E c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "TreePredictions" device_type: "CPU"') for unknown op
E c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "UpdateFertileSlots" device_type: "CPU"') for unknown

Source | Train? | Layer description                | Output size
-------+--------+----------------------------------+---------------
       |        | input                            | (?, 416, 416, 3)
 Load  |  Yep!  | conv 3x3p1_1  +bnorm  leaky      | (?, 416, 416, 32)
 Load  |  Yep!  | maxp 2x2p0_2                     | (?, 208, 208, 32)
 Load  |  Yep!  | conv 3x3p1_1  +bnorm  leaky      | (?, 208, 208, 64)
 Load  |  Yep!  | maxp 2x2p0_2                     | (?, 104, 104, 64)
 Load  |  Yep!  | conv 3x3p1_1  +bnorm  leaky      | (?, 104, 104, 128)
 Load  |  Yep!  | conv 1x1p0_1  +bnorm  leaky      | (?, 104, 104, 64)
 Load  |  Yep!  | conv 3x3p1_1  +bnorm  leaky      | (?, 104, 104, 128)
 Load  |  Yep!  | maxp 2x2p0_2                     | (?, 52, 52, 128)
 Load  |  Yep!  | conv 3x3p1_1  +bnorm  leaky      | (?, 52, 52, 256)
 Load  |  Yep!  | conv 1x1p0_1  +bnorm  leaky      | (?, 52, 52, 128)
 Load  |  Yep!  | conv 3x3p1_1  +bnorm  leaky      | (?, 52, 52, 256)
 Load  |  Yep!  | maxp 2x2p0_2                     | (?, 26, 26, 256)
 Load  |  Yep!  | conv 3x3p1_1  +bnorm  leaky      | (?, 26, 26, 512)
 Load  |  Yep!  | conv 1x1p0_1  +bnorm  leaky      | (?, 26, 26, 256)
 Load  |  Yep!  | conv 3x3p1_1  +bnorm  leaky      | (?, 26, 26, 512)
 Load  |  Yep!  | conv 1x1p0_1  +bnorm  leaky      | (?, 26, 26, 256)
 Load  |  Yep!  | conv 3x3p1_1  +bnorm  leaky      | (?, 26, 26, 512)
 Load  |  Yep!  | maxp 2x2p0_2                     | (?, 13, 13, 512)
 Load  |  Yep!  | conv 3x3p1_1  +bnorm  leaky      | (?, 13, 13, 1024)
 Load  |  Yep!  | conv 1x1p0_1  +bnorm  leaky      | (?, 13, 13, 512)
 Load  |  Yep!  | conv 3x3p1_1  +bnorm  leaky      | (?, 13, 13, 1024)
 Load  |  Yep!  | conv 1x1p0_1  +bnorm  leaky      | (?, 13, 13, 512)
 Load  |  Yep!  | conv 3x3p1_1  +bnorm  leaky      | (?, 13, 13, 1024)
 Load  |  Yep!  | conv 3x3p1_1  +bnorm  leaky      | (?, 13, 13, 1024)
 Load  |  Yep!  | conv 3x3p1_1  +bnorm  leaky      | (?, 13, 13, 1024)

F c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\cuda\cuda_driver.cc:94] Check failed: s.ok() could not find cuDevicePrimaryCtxSetFlags in libcuda DSO; dlerr
or: cuDevicePrimaryCtxSetFlags not found

i'm using python 3.5, windows 7(64bit), tensorflow (v1.0 installed via pip) with gpu support, cuda v8, cudnn v5
i'm having NVidia GeForce 210 video card, have installed latest graphic driver

note:
i also tried cuda v5 toolkit, but somehow tensorflow is still looking for v8 cuda binaries ..
i installed tensorflow cpu only version, it works. but 0.4fps

Thanks

Is there any way to use this without OpenCV?

I'm trying to use this but I'd really rather use this without OpenCV - is that possible?

data augmentation

Hi：
This is a fantastic project and I have read your code. I got confused when I read the part of preporcess in
Test.py. The image were transformed using scale, translation, however, the annotations were not transformed correctly.

fix(obj,dims,scale, offs):
    for i in range(1, 5):
        dim = dims[(i + 1) % 2]
        off = offs[(i + 1) % 2]
        obj[i] = int(obj[i]*scale-off)
        obj[i] = max(min(obj[i], dim), 0)

In the code above, you just compare the coords of transformed bounding box with the original dimension of the image, which I think is wrong. you should compare with the transformed image dimension.

Error in camera

I am trying to use the camera demo and I got this error

Running entirely on CPU
Finished in 1.87715506554s

Press [ESC] to quit demo
Traceback (most recent call last):
  File "./flow", line 49, in <module>
    tfnet.camera(FLAGS.demo)
  File "/home/moh/Downloads/darkflow/net/help.py", line 78, in camera
    })[0], frame, False))
  File "/home/moh/Downloads/darkflow/net/yolo/test.py", line 106, in postprocess
    if max_prob > _thresh[label]:
KeyError: 'aeroplane'

Fine tuning error

I am trying to Fine tune the darkflow on my data that contain 2 classes:
1- I created the label.txt file
2- I modified process.py an in #32
3- I downloaded yolo.weights It has size of 789.3 MB
4- I copied the yolov1.cfg into yolov1-2c.cfg and modified only this parameter classes=2
but the problem begin to appear when I use this command
./flow --model cfg/v1.1/yolov1-2c.cfg --load bin/yolo.weights
I get this error

Parsing ./cfg/yolo.cfg
Parsing cfg/v1.1/yolov1-2c.cfg
Loading bin/yolo.weights ...
Traceback (most recent call last):
  File "./flow", line 42, in <module>
    tfnet = TFNet(FLAGS)
  File "/home/moh/Documents/darkflow/net/build.py", line 34, in __init__
    darknet = Darknet(FLAGS)
  File "/home/moh/Documents/darkflow/dark/darknet.py", line 27, in __init__
    self.load_weights()
  File "/home/moh/Documents/darkflow/dark/darknet.py", line 82, in load_weights
    wgts_loader = loader.create_loader(*args)
  File "/home/moh/Documents/darkflow/utils/loader.py", line 104, in create_loader
    return load_type(path, cfg)
  File "/home/moh/Documents/darkflow/utils/loader.py", line 18, in __init__
    self.load(*args)
  File "/home/moh/Documents/darkflow/utils/loader.py", line 76, in load
    walker.offset, walker.size)
AssertionError: expect 269862452 bytes, found 789312988

What I am missing here ?

Fine tuning

I have an error while trying to fine tune your model on my data.

I followed your instructions :
First I wrote two classes only for example good and bad in Labels.txt file
and ran the clean.py which gave me parsed.yolotf

for designing the network:
I copied the yolo-tiny.cfg and renamed it to yolo-2c.cfg and changed the number of classes classes=2 and output= 588 according to the equation in the paper

for Initializing the weights :
I used yolo.tiny.weights file to initialize my network weights and generated new file
yolo-2c.weights using this command
python genw.py tiny 2c

for Flowing the graph:
I used
python main.py --train --model 2c --noload
and got this error

Problem with batch_norm()

Hey, I tried loading various models and I always get this error. For example when I run: ./flow --model cfg/v1.1/tiny-yolov1.cfg

I get the following.

Parsing cfg/v1.1/tiny-yolov1.cfg
Loading None ...
Finished in 7.70092010498e-05s

Building net ...
Source | Train? | Layer description | Output size
-------+--------+----------------------------------+---------------
Traceback (most recent call last):
File "./flow", line 41, in
tfnet = TFNet(FLAGS)
File "/home/nataniel/Documents/gatech/research/darkflow/net/build.py", line 50, in init
self.build_forward()
File "/home/nataniel/Documents/gatech/research/darkflow/net/build.py", line 70, in build_forward
state = op_create(*args)
File "/home/nataniel/Documents/gatech/research/darkflow/net/ops/init.py", line 25, in op_create
return op_typeslayer_type
File "/home/nataniel/Documents/gatech/research/darkflow/net/ops/baseop.py", line 42, in init
self.forward()
File "/home/nataniel/Documents/gatech/research/darkflow/net/ops/convolution.py", line 66, in forward
temp = self.batchnorm(self.lay, temp)
File "/home/nataniel/Documents/gatech/research/darkflow/net/ops/convolution.py", line 78, in batchnorm
is_training = layer.h['is_training'])
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/framework/python/ops/arg_scope.py", line 177, in func_with_args
return func(*args, **current_args)
TypeError: batch_norm() got an unexpected keyword argument 'initializers'

I am using TF version 0.12.0-rc0 and python 2.7

Thanks !

train error

I am trying to train the darkflow :
python3 ./flow.py --model cfg/tiny-yolo-4c.cfg --train --trainer adam
I get this error
Parsing cfg/tiny-yolo-4c.cfg
Loading None ...
Finished in 4.506111145019531e-05s

Building net ...
Source | Train? | Layer description | Output size
-------+--------+----------------------------------+---------------
| | input | (?, 416, 416, 3)
Init | Yep! | conv 3x3p1_1 +bnorm leaky | (?, 416, 416, 16)
Load | Yep! | maxp 2x2p0_2 | (?, 208, 208, 16)
Init | Yep! | conv 3x3p1_1 +bnorm leaky | (?, 208, 208, 32)
Load | Yep! | maxp 2x2p0_2 | (?, 104, 104, 32)
Init | Yep! | conv 3x3p1_1 +bnorm leaky | (?, 104, 104, 64)
Load | Yep! | maxp 2x2p0_2 | (?, 52, 52, 64)
Init | Yep! | conv 3x3p1_1 +bnorm leaky | (?, 52, 52, 128)
Load | Yep! | maxp 2x2p0_2 | (?, 26, 26, 128)
Init | Yep! | conv 3x3p1_1 +bnorm leaky | (?, 26, 26, 256)
Load | Yep! | maxp 2x2p0_2 | (?, 13, 13, 256)
Init | Yep! | conv 3x3p1_1 +bnorm leaky | (?, 13, 13, 512)
Load | Yep! | maxp 2x2p0_1 | (?, 13, 13, 512)
Init | Yep! | conv 3x3p1_1 +bnorm leaky | (?, 13, 13, 1024)
Init | Yep! | conv 3x3p1_1 +bnorm leaky | (?, 13, 13, 1024)
Init | Yep! | sele 1x1p0_1 linear | (?, 13, 13, 45)
-------+--------+----------------------------------+---------------
Running entirely on CPU
cfg/tiny-yolo-4c.cfg loss hyper-parameters:
H = 13
W = 13
box = 5
classes = 4
scales = [1.0, 5.0, 1.0, 1.0]
Traceback (most recent call last):
File "./flow.py", line 42, in
tfnet = TFNet(FLAGS)
File "/home/qkj/projects/darkflow/net/build.py", line 51, in init
self.setup_meta_ops()
File "/home/qkj/projects/darkflow/net/build.py", line 94, in setup_meta_ops
if self.FLAGS.train: self.build_train_op()
File "/home/qkj/projects/darkflow/net/help.py", line 15, in build_train_op
self.framework.loss(self.out)
File "/home/qkj/projects/darkflow/net/yolov2/train.py", line 61, in loss
coords = tf.concat(3, [adjusted_coords_xy, adjusted_coords_wh])
File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/ops/array_ops.py", line 1053, in concat
dtype=dtypes.int32).get_shape(
File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/framework/ops.py", line 651, in convert_to_tensor
as_ref=False)
File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/framework/ops.py", line 716, in internal_convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/framework/constant_op.py", line 176, in _constant_tensor_conversion_function
return constant(v, dtype=dtype, name=name)
File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/framework/constant_op.py", line 165, in constant
tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape, verify_shape=verify_shape))
File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/framework/tensor_util.py", line 367, in make_tensor_proto
_AssertCompatible(values, dtype)
File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/framework/tensor_util.py", line 302, in _AssertCompatible
(dtype.name, repr(mismatch), type(mismatch).name))
TypeError: Expected int32, got list containing Tensors of type '_Message' instead.

input images

Thank you for your great repository!
My question is not directly related to your repository, but what is darkflow & darknet input images?
Ex. input images should be normalized(0-255 -> 0-1)?

Now, I'm trying to use yolo model made by this repository on Android(https://github.com/tensorflow/tensorflow/tree/master/tensorflow/examples/android/), but it seems to be something wrong...
My android output confidence is always nearly 1.

Sorry to bother you.

tensorflow-yolo demo gives negative confidence

Thank you for your repository!
I used this repo to make .pb file from .weights.
And then I put my .pb file into tensorflow-yolo detector demo(https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/android/src/org/tensorflow/demo/TensorFlowYoloDetector.java).
But this code gave me negative confidence in line 236!
If you know something about it, could you inform me?
What is the relationship between "threshold" in darkflow and "confidence, maxclass" in the above code?
Thank you

YOLO 9000

Hello guys!

Thanks for the amazing work!

I'm looking for the YOLO9000 model that can classify 9000 categories.

I could not find the YOLO 9000 cfg.

Can anyone help please?

> find . -name '*.cfg' -exec cat {} \; | grep "classes"
classes=4
classes=20
classes=80
classes=2
classes=4
classes=20
classes=20
classes=4
classes=4
classes=20
classes=4
classes=2
classes=80
classes=4
classes=20
classes=80
classes=20
classes=20
classes=80

I was expecting something like classes=9000

How to load pretrained weight of a partial network.

This is the answer from the author (Thanks for author's patient reply)

1. Flatten all of your convolutional weight and concatenate them into a big long vector, in the order of lower layer first (input layer first).
2. Initialize the dense layers, flatten and append them into the big long vector as well.
3. Prepend 4 random bytes to your big long vector
4. Dump the resulting big long vector to a binary file, say yolo-myConfig.weights

These can be done in a few Python lines of code. After that, you can now train on your pre-trained weight using
main.py --model myConfig --train --noload

(suggestion) Generate text file of predicted bounding boxes

HI, Trinh,

Thank you for providing neatly organized codes to use YOLO2 in tensorflow.

I wanted to generate text file of predicted bounding boxes, with original YOLO2 implementation;
https://groups.google.com/forum/#!searchin/darknet/text$20output%7Csort:relevance/darknet/ylEWe3JUKrE/OG--eSmiCwAJ

But I gave up due to my poor knowledge on C. Then I found this implementation and made it. I did not have time to analyze your implementation, so my way is very simple. I just added a few lines. Especially something like the below around here:

darkflow/net/yolov2/test.py

Line 86 in 718e756

thick = int((h+w)/300)

f.write("%s,%s,%d,%d,%d,%d\n"%(im,label,left,right,top,bot))

I guess it's easy for you to add a flag to output json or text file instead of annotated images. When you have time, can you please do that?

Regards,
Satoshi

yolo-tiny.cfg vs. yolo-baby.cfg

Not really an issue but, I wonder what is the difference between the yolo-tiny.cfg vs. yolo-baby.cfg and whether yolo-baby.cfg is supposed to work in darktf revision c963245 ?

Is yolo-baby.cfg an older style Darknet configuration?

Did you test the speed in IOS. Thanks!

KeyError: 'thresh'

$ ./flow --model yolo-voc.cfg --load yolo-voc.weights --demo camera
...
Load | Yep! | concat [26, 24] | (?, 13, 13, 3072)
Load | Yep! | conv 3x3p1_1 +bnorm leaky | (?, 13, 13, 1024)
Load | Yep! | conv 1x1p0_1 linear | (?, 13, 13, 125)
-------+--------+----------------------------------+---------------
Running entirely on CPU
I tensorflow/core/common_runtime/gpu/gpu_device.cc:885] Found device 0 with properties:
name: GeForce GTX 1080
major: 6 minor: 1 memoryClockRate (GHz) 1.8475
pciBusID 0000:01:00.0
Total memory: 7.92GiB
Free memory: 7.47GiB
I tensorflow/core/common_runtime/gpu/gpu_device.cc:906] DMA: 0
I tensorflow/core/common_runtime/gpu/gpu_device.cc:916] 0: Y
I tensorflow/core/common_runtime/gpu/gpu_device.cc:975] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1080, pci bus id: 0000:01:00.0)
WARNING:tensorflow:From /home/rob/darkflow/net/build.py:98: all_variables (from tensorflow.python.ops.variables) is deprecated and will be removed after 2017-03-02.
Instructions for updating:
Please use tf.global_variables instead.
Finished in 8.21734499931s

VIDIOC_QUERYMENU: Invalid argument
VIDIOC_QUERYMENU: Invalid argument
VIDIOC_QUERYMENU: Invalid argument
VIDIOC_QUERYMENU: Invalid argument
VIDIOC_QUERYMENU: Invalid argument
VIDIOC_QUERYMENU: Invalid argument
VIDIOC_QUERYMENU: Invalid argument
Press [ESC] to quit demo
Traceback (most recent call last):
File "./flow", line 49, in
tfnet.camera(FLAGS.demo)
File "/home/rob/darkflow/net/help.py", line 78, in camera
})[0], frame, False))
File "/home/rob/darkflow/net/yolov2/test.py", line 23, in postprocess
threshold = meta['thresh']
KeyError: 'thresh'

Not sure why 'thresh' is missing. Looking into it.

what is the lowest loss value can reach?

hi, I have trained a yolo-small model to step 4648, but most of loss values are greater than 1.0, and the result of test is not very well. I want to know how well can loss value be, and could you please show some key parameters when training, e.g learning rate, training time, the final loss value, and so on.

I train the model on iMac(4 GHz Inter Core i7, 16GB memory), CPU mode.

thank you!

About Data.py 11 tensor

Thanks for this awesome work!
Just a little bit confused about the 11 tensors in data.py

new = [ [probs], [confs1], [confs2],
             [coord],[upleft], [botright], [proid],
             [conid1], [conid2], [cooid1], [cooid2] ]

probs seems to represent Pr(class | obj )
confs1 seems to represent the confidence in box 1 -- Pr(obj)
confs2 seems to represent the confidence in box 2 -- Pr(obj)

According to the original paper, the rest of items seems going to represent [x,y,w,h] in Box1 and Box2.
However, I have a hard time to figure out it.

Could you give me some hints?
Thank you !

question about padding in conv2d

In the file net/ops/convolution.py:

temp = tf.pad(self.inp.out, [[0, 0]] + pad + [[0, 0]])
temp = tf.nn.conv2d(temp, self.lay.w['kernel'], padding = 'VALID',  name = self.scope, strides = [1] + [self.lay.stride] * 2 + [1])

Is this equivalent to the 'same' padding, tf.nn.conv2d(..., padding ='SAME')?

Can I use darkflow to convert darknet.cfg to pb ? (or other classify models)

Hi thtrieu,
I meet a problem, when i use darkflow to convert darknet.cfg & darknet.weights to pb.

The following is my log:

./flow --model …/darknet/cfg/darknet.cfg --load …/darknet/darknet.weights --savepb --verbalise=True
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcurand.so locally
/home/hankwu/DeepLearning/darkflow/dark/darknet.py:54: UserWarning: ./cfg/darknet.cfg not found, use …/darknet/cfg/darknet.cfg instead
cfg_path, FLAGS.model))
Parsing …/darknet/cfg/darknet.cfg
Loading …/darknet/darknet.weights …
Successfully identified 29293936 bytes
Finished in 0.00332188606262s
Traceback (most recent call last):
File “./flow”, line 42, in
tfnet = TFNet(FLAGS)
File “/home/hankwu/DeepLearning/darkflow/net/build.py”, line 41, in init
self.framework = create_framework(*args)
File “/home/hankwu/DeepLearning/darkflow/net/framework.py”, line 43, in create_framework
return this(meta, FLAGS)
File “/home/hankwu/DeepLearning/darkflow/net/framework.py”, line 8, in init
self.constructor(*args)
AttributeError: ‘framework’ object has no attribute ‘constructor’

Many Thanks,
Hank

Saved graph returns non-deterministic results

I've converted a YOLO/darknet graph using darkflow --savepb option.
I am loading it in another TensorFlow project and I manage to run it and get some data, which looks ok.
However, it seems that every time that I run it, I get slightly different results.
Maybe it is related to the dropout layer which has some randomness in it? I've noticed that in "test" mode, darkflow is consistent, but could not find what I am missing in my own code that causes it to be deterministic in darkflow.
Any advice?

yolov2 error

Tried a v2 trained model (own dataset) from the C yolov2 project, got following error:
----Traceback (most recent call last):
File "./flow", line 42, in
tfnet = TFNet(FLAGS)
File "/home/ai/darkflow/net/build.py", line 34, in init
darknet = Darknet(FLAGS)
File "/home/ai/darkflow/dark/darknet.py", line 16, in init
src_parsed = self.parse_cfg(self.src_cfg, FLAGS)
File "/home/ai/darkflow/dark/darknet.py", line 66, in parse_cfg
else: new = create_darkop(*info)
File "/home/ai/darkflow/dark/darkop.py", line 61, in create_darkop
return op_class(ltype, num, *args)
File "/home/ai/darkflow/dark/layer.py", line 19, in init
size = np.prod(shp)
File "/home/tensorflow/local/lib/python2.7/site-packages/numpy/core/fromnumeric.py", line 2539, in prod
out=out, **kwargs)
File "/home/tensorflow/local/lib/python2.7/site-packages/numpy/core/_methods.py", line 35, in _prod
return umr_prod(a, axis, dtype, out, keepdims)
TypeError: cannot perform reduce with flexible type

Take a quick look at dark/layer.py and could not figure out quickly what went wrong. Running with latest Tensorflow 0.12.

Same model/weight files work very well with YoloV2 C project. Use the command from this website: ./flow --test --model cfg-file --load weight-files

Initialize weights

I'm trying to train yolo v1.1 on my own data of two classes
I made a copy of yolov1.cfg file and changed the output of the connected layer to 833
according to this formula
(5 x num + number_of_classes) x side x side
where num = 3 and side = 7 in the cfg file of yolov1
I downloaded yolo.weights file of size 789 MB as mentioned in the README.md file
but I got this error

p.s. I used an old version of this repo before, to train my data on a yolov1.0 model successfully
and it had genw.py file to initialize the weights of the 2 classes model with weights of a pre-trained one

but I don't know how to initialize weights now