satojkovic / deeplogo

A brand logo detection system using tensorflow object detection API.

[Update]

DeepLogo2 with DETR is available

DeepLogo

A brand logo detection system using Tensorflow Object Detection API.

Description

Tensorflow Object Detection API is the easy to use framework for creating a custom deep learning model that solves object detection problems.

If you already have your own dataset, you can simply create a custom model with sufficient accuracy using a collection of detection models pre-trained on COCO, KITTI, and OpenImages dataset.

DeepLogo provides training and evaluation environments of Tensorflow Object Detection API for creating a brand logo detection model.

Detection Results

These are some detection results by DeepLogo.

Training DeepLogo

DeepLogo uses SSD as a backbone network and fine-tunes pre-trained SSD released in the tensorflow/models repository.

In order to use that pre-trained model, setting up the tensorflow/models repository first.

1. Clone the tensorflow/models repository and download the pre-trained model from model zoo.

   $ git clone https://github.com/tensorflow/models.git
   $ cd models/research/object_detection
   $ wget http://download.tensorflow.org/models/object_detection/ssd_inception_v2_coco_2018_01_28.tar.gz
   $ tar zxvf ssd_inception_v2_coco_2018_01_28.tar.gz

   Tensorflow Object Detection API depends on many other libraries. For detailed steps to setup, please follow the official installation instruction.

2. Clone the DeepLogo repository.

   $ git clone https://github.com/satojkovic/DeepLogo.git

3. Download the flickr logos 27 dataset from here.

   The flickr logos 27 dataset contains 27 classes of brand logo images downloaded from Flickr. The brands included in the dataset are: Adidas, Apple, BMW, Citroen, Coca Cola, DHL, Fedex, Ferrari, Ford, Google, Heineken, HP, McDonalds, Mini, Nbc, Nike, Pepsi, Porsche, Puma, Red Bull, Sprite, Starbucks, Intel, Texaco, Unisef, Vodafone and Yahoo.

   $ cd DeepLogo
   $ wget http://image.ntua.gr/iva/datasets/flickr_logos/flickr_logos_27_dataset.tar.gz
   $ tar zxvf flickr_logos_27_dataset.tar.gz
   $ cd flickr_logos_27_dataset
   $ tar zxvf flickr_logos_27_dataset_images.tar.gz
   $ cd ../

4. Preprocess

   The flickr logos 27 dataset contains an annotation file for training. This file includes not valid annotations such as an empty size bounding box. Therefore these annotations are removed in this preprocess step, then class names are converted into class numbers and generate two preprocessed files. These two files are used to generate tfrecord files.

   $ cd DeepLogo
   $ python preproc_annot.py

5. Generate tfrecord files.

   The Tensorflow Object Detection API expects data to be in the TFRecord format. Run the following command to convert from preprocessed files into TFRecords.

   $ python gen_tfrecord.py --train_or_test train --csv_input flickr_logos_27_dataset/flickr_logos_27_dataset_training_set_annotation_cropped.txt --img_dir flickr_logos_27_dataset/flickr_logos_27_dataset_images --output_path train.tfrecord
   $ python gen_tfrecord.py --train_or_test test --csv_input flickr_logos_27_dataset/flickr_logos_27_dataset_test_set_annotation_cropped.txt --img_dir flickr_logos_27_dataset/flickr_logos_27_dataset_images --output_path test.tfrecord

6. Training

   The Tensorflow Object Detection API has a python script for training called train.py. This script needs two arguments --pipeline_config_path and --train_dir. (see below)

   DeepLogo assumes that the current directory is under the DeepLogo directory and also the path of pre-trained SSD and tfrecord is the relative path from DeepLogo (these paths are written in ssd_inception_v2.config). Therefore create a symbolic link to the directory of tensorflow/models/research/object_detection/ssd_inception_v2_coco_2018_01_28 first, then run the training script.

   $ OBJECT_DETECTION_API_DIR={path to tensorflow/models/research/object_detection}
   $ ln -s ${OBJECT_DETECTION_API_DIR}/ssd_inception_v2_coco_2018_01_28 ssd_inception_v2_coco_2018_01_28
   $ python ${OBJECT_DETECTION_API_DIR}/legacy/train.py --logtostderr --pipeline_config_path=ssd_inception_v2_coco.config --train_dir=training

   Note: DeepLogo doesn't work in Tensorflow 2.0. When you try to train DeepLogo, checkout 5ba3c3f5 of tensorflow/models.

Testing

1. Exporting a trained model for inference

   For testing a model, you should export it to a Tensorflow graph proto first.

   $ STEPS={the number of steps when the model is saved}
   $ python ${OBJECT_DETECTION_API_DIR}/export_inference_graph.py --input_type=image_tensor --pipeline_config_path=ssd_inception_v2_coco.config --trained_checkpoint_prefix=model.ckpt-${STEPS} --output_directory=logos_inference_graph

   Alternatively, you can download a trained model from GoogleDrive!

2. Inference

   Run the following command. The results of logo detection are saved in --output_dir.

   $ python logo_detection.py --model_name logos_inference_graph/ --label_map flickr_logos_27_label_map.pbtxt --test_annot_text flickr_logos_27_dataset/flickr_logos_27_dataset_test_set_annotation_cropped.txt --test_image_dir flickr_logos_27_dataset/flickr_logos_27_dataset_images --output_dir detect_results

Evaluation

Before evaluating the trained model saved in training directory, edit the num_examples field in training/pipeline.config file.

The num_examples field represents the number of test images which is equal to number of lines present in a flickr_logos_27_dataset_test_set_annotation_cropped.txt file.

$ wc -l flickr_logos_27_dataset/flickr_logos_27_dataset_test_set_annotation_cropped.txt
     438 flickr_logos_27_dataset/flickr_logos_27_dataset_test_set_annotation_cropped.txt

eval_config: {
  num_examples: 438
  # Note: The below line limits the evaluation process to 10 evaluations.
  # Remove the below line to evaluate indefinitely.
  max_evals: 10
}

Then start evaluation process by using eval.py provided within tensorflow/models repository.

$ python ${OBJECT_DETECTION_API_DIR}/legacy/eval.py --logtostderr --checkpoint_dir=training --eval_dir=eval --pipeline_config_path=training/pipeline.config

After a while you will get evaluation results. If you want to check the results visually, open tensorboard in your browser.

$ tensorboard --logdir=eval/

License

MIT

deeplogo's Issues

Wrong parameter inside call to training function

There is a mistake in calling to trainig function:
$ python ${OBJECT_DETECTION_API_DIR}/legacy/train.py --logtostderr --pipeline_config_path=ssd_inception_v2.config --train_dir=training
If I understand correctly, should be:
$ python ${OBJECT_DETECTION_API_DIR}/legacy/train.py --logtostderr --pipeline_config_path=ssd_inception_v2_coco.config --train_dir=training
The coco postfix is missing

Bounding Boxes for Logos

Hi, I have a question about the output images in the 'detect_results' directory. I see that there is both a bounding box around the logo and a confidence score for which brand the logo belongs to. I want to ask if the bounding box is determined during the testing process, or is the bounding box simply drawn using the coordinates from the annotation.txt file? I am asking this because I am wondering if I run DeepLogo on my own dataset, will it be able to identify and draw the bounding boxes automatically, or do I need to provide these in a .txt file? I am less interested in the classification, but more interested in potentially using DeepLogo to automate the process of locating a logo within an image. Is this possible? Thanks.

model

hello @satojkovic , can you please provide us with the full exported folder ?
i'm working on a logo detection project and i would really appreciate it if you upload it so i can use it as a backbone

Unable to process data from flickr_logos_27_dataset\flickr_logos_27_dataset_cropped_augmented_images\Lift\train.pickle

Hello !

I've tried to use this wonderful program to detect my own logos. For now i have just two logos wich represent a lift and an escalator. I change the files "flickr_logos_27_dataset_training_set_annotation.txt" and "flickr_logos_27_dataset_query_set_annotation.txt" to match with my samples. I also modified the python's file to set the number of classes to 2.

The crop_and_aug.py works well but when i launch gen_train_valid_test.py, after processes all the classes, I got the following error :
"Unable to process data from flickr_logos_27_dataset\flickr_logos_27_dataset_cropped_augmented_images\Lift\train.pickle : could not broadcast input array from shape (576,32,64,3) into shape (2500,32,64,3)
Traceback (most recent call last):
File "gen_train_valid_test.py", line 192, in
main()
File "gen_train_valid_test.py", line 178, in main
train_datasets, TRAIN_SIZE, VALID_SIZE)
File "gen_train_valid_test.py", line 123, in merge_datasets
valid_dataset[start_v:end_v, :, :, :] = valid_logo
ValueError: could not broadcast input array from shape (576,32,64,3) into shape (2500,32,64,3)"

I would be greatful if you know how i can solve this issue :) Thanks !

[ EDIT 24/11/2017]
After few hours looking for the origin of the issue, and i found that it come from my edit on the "flickr_logos_27_dataset_training_set_annotation.txt", and i don't really how to fill it properly.
Do you know a simple way to easily make your own dataset ?

Regards
Seevoid

Cant able to download dataset

http://image.ntua.gr/iva/datasets/flickr_logos/flickr_logos_27_dataset.tar.gz
The dataset download link is not responding..

FileNotFoundError: [Errno 2] No such file or directory: 'deep_logo.pickle'

can u help me, please
when i run code .
error -> FileNotFoundError: [Errno 2] No such file or directory: 'deep_logo.pickle'

Class Label missing

Hi SatojKovic,
I followed your steps and was able to generate model successfully, but I wanted to use this model in my iOS application for learning purpose. So, I converted the model into .pb and then that into coreml. But while doing that, I can't find the classLabel file. Though class names are present in common.py. So, am I missing some thing? Can you please help me out here.
Thanks

Broken link to tf object detection API

The link produces a 404 not found error

problem in importing common

Please help

AttributeError: module 'common' has no attribute 'CLASS_NAME'

Input Images to Selective Search

When i provide input Image from folder provide query_set_results,it shows AssertionError: 3ch image is expected.Do i need to resize or any change in Image before providing input to program?

feedback

Hi
Thanks for the code.
I trained and tested it, but the outputs are not matching, please check.

gopi@gp:~/tensorflow/tensorflow/logo/DeepLogo-master$ python test_deep_logo_cnn.py
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcublas.so.8.0 locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcudnn.so.5.1.5 locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcufft.so.8.0 locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcurand.so.8.0 locally
('Test image:', 'flickr_logos_27_dataset/flickr_logos_27_dataset_cropped_augmented_images/Citroen/test/3577850173_Citroen_5_80_r-14.jpg')
I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:925] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
I tensorflow/core/common_runtime/gpu/gpu_device.cc:951] Found device 0 with properties:
name: GeForce GTX 1070
major: 6 minor: 1 memoryClockRate (GHz) 1.835
pciBusID 0000:01:00.0
Total memory: 7.92GiB
Free memory: 7.29GiB
I tensorflow/core/common_runtime/gpu/gpu_device.cc:972] DMA: 0
I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] 0: Y
I tensorflow/core/common_runtime/gpu/gpu_device.cc:1041] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1070, pci bus id: 0000:01:00.0)
Model restored
('Class name:', 'Nbc')

Issue with test_deep_logo_cnn

First of all, great work, easy to run and understand! Thanks a bunch!

I having few problems with testing process.

1. First, the code is written so that it chooses a random image from random test folder, so whenever i change it to select i) select only one test folder and then i give it a image file, it gives wrong answer. for example, i made it choose apple test folder, then i gave it an image of Ford, the answer still says APPLE. (pics attached)
   

2. Why did u choose to set probability to max? I mean i want to understand your logic.

3. Can i give it images outside of test folders to test it out?
   I mean if i give it a random image from internet, will it recognize it?

4. Output has a lot of weird 'Output", i mean it shows alot of weird info. if u can help me clear it up?
   

It would greatly help me if you can remove my confusions.

P.S: I am working on Windows 10 64 bit with Anaconda to run the test_deep_logo_cnn, as I cant seem to find a way to run scipy (on 64 bit python and pyhton 3.5 ) and tensorflow together.

crop_and_aug.py giving error

Hi, I am trying crop and augment the flickr_logos_27_dataset using below command

(tensorflow)rakashi@pavan:~/PycharmProjects/DeepLogo-master$ python crop_and_aug.py

I am unable to identify what is the problem, Ia musing the libraries list as

Pillow (2.3.0)
numpy (1.13.1)

Traceback (most recent call last):
File "/home/rakashi/PycharmProjects/DeepLogo-master/crop_and_aug.py", line 313, in
main()
File "/home/rakashi/PycharmProjects/DeepLogo-master/crop_and_aug.py", line 306, in main
crop_and_aug_with_none(annot_train)
File "/home/rakashi/PycharmProjects/DeepLogo-master/crop_and_aug.py", line 254, in crop_and_aug_with_none
crop_and_aug(annot_train)
File "/home/rakashi/PycharmProjects/DeepLogo-master/crop_and_aug.py", line 200, in crop_and_aug
shifted_ims, shifted_suffixes = aug_pos(annot, im)
File "/home/rakashi/PycharmProjects/DeepLogo-master/crop_and_aug.py", line 90, in aug_pos
cropped_im.close()
File "/usr/lib/python2.7/dist-packages/PIL/Image.py", line 528, in getattr
raise AttributeError(name)
AttributeError: close

value of the loss function

What was the the value of the total loss function at training process termination, in your case?

Given an input image, output if logo exists

Hi,
I was trying to run the code step by step. The third step "python train_deep_logo_cnn.py" gives me a code dump.
But anyways, what I need is given my set of logos for a special brand, I train the model and when I run, it specifies if that logo exists in a given image. Is it possible to do that?

Allow batch logo detection

Include the functionality to detect logos in a batch of images instead of a single image a time

How to create deep_logo.pickle file ?

flickr_logos_27_dataset/flickr_logos_27_dataset_training_set_annotation.txt not found.

when i run gen_bg_class.py，it gaves this error:
OSError: flickr_logos_27_dataset/flickr_logos_27_dataset_training_set_annotation.txt not found.

Does anyone know this reason, Thanks!

ValueError: could not broadcast input array from shape (5662,32,64,3) into shape (17500,32,64,3)

I am glad to find your code. But here is some error.
in merge_datasets method: ValueError: could not broadcast input array from shape (5662,32,64,3) into shape (17500,32,64,3).
I didn't change any code or any data, images or text file you given in github.
I fixed one error, here it is:
def do_train_test_split():
class_names = [cls for cls in os.listdir(common.CROPPED_AUG_IMAGE_DIR)]
for class_name in class_names:
if class_name != ".DS_Store": #that file was inside class names in my mac.

Can you help me to fix the error on top. Check my issue and guide me. Thank u.

Data set issue

http://image.ntua.gr/iva/datasets/flickr_logos/flickr_logos_27_dataset.tar.gz

This dataset cant be download

Precision/recall

Hello,
I'm currently iterating over DeepLogo with some more recent object detection models, and was wondering if you could share the global average precision/recall you obtained on the flickr logos 27 dataset after training DeepLogo?

ArgumentError: argument --image_width: conflicting option string: --image_width

when I run test_deep_logo_cnn.py -> error

ArgumentError: argument --image_width: conflicting option string: --image_width

Someone help me pleae
thanhk!

Classification after training model

I have followed same instructions to train model and saved it..but when i use detect_logo.py for detecting logo with test images for most of the images it gives me "Background"..Please guide me what is wrong with my training.

IO error no such file as Deep_Logo.pickle

test_deep_logo_cnn.py

Hi @satojkovic , i have an one doubt about the test file. whether it is test only the random of file from the image in dataset or test the classification is done by whole test images in data-sets?

Can you publish accuracy results?

Unable to process data in gen_train_valid_test.py script

Hey guys,

i have problems running the gen_train_valid_test.py script. Seems to be a shape issue while loading the pickle data.

I used the flickr27_logos dataset. The crop and augmentation script worked just fine.

I get following exception:

('Unable to process data from', 'flickr_logos_27_dataset/flickr_logos_27_dataset_cropped_augmented_images/Ford/test.pickle', ':', ValueError('could not broadcast input array from shape (1387,32,64,3) into shape (1851,32,64,3)',)) Traceback (most recent call last): File "gen_train_valid_test.py", line 191, in <module> main() File "gen_train_valid_test.py", line 177, in main train_datasets, TRAIN_SIZE, VALID_SIZE) File "gen_train_valid_test.py", line 128, in merge_datasets train_dataset[start_t:end_t, :, :, :] = train_logo ValueError: could not broadcast input array from shape (1387,32,64,3) into shape (1851,32,64,3)

It processes all classes and failed when it is loading the single pickle files inside the class directories i think.

Has anyone an idea how to fix this issue?

best regards and thanks
Frey

Documentation

Can you please add step by step documentation on setting your code up and running custom training job ?

DeepLogo's training/testing performance on the given database?

This is a very interesting project on logo recognition based deep learning. It would be great if author can include the classification results on the given logo image datasets.

Thanks

Will you release trained model?

Thanks for the great work!
Would you plan to release your trained model on those logo dataset?

No such file or directory: '-f'

Thank you for your work.
Icrop_and_aug.py - successfully
2. gen_train_valid_test.py - successfully
3. train_deep_logo_cnn.py : - I've got an error

FileNotFoundError Traceback (most recent call last)
in ()
254
255 if name == 'main':
--> 256 main()

in main()
113 def main():
114 if len(sys.argv) > 1:
--> 115 f = np.load(sys.argv[1])
116
117 # f.files has unordered keys ['arr_8', 'arr_9', 'arr_6'...]

~.conda\envs\AnacondaEnv\lib\site-packages\numpy\lib\npyio.py in load(file, mmap_mode, allow_pickle, fix_imports, encoding)
368 own_fid = False
369 if isinstance(file, basestring):
--> 370 fid = open(file, "rb")
371 own_fid = True
372 elif is_pathlib_path(file):

FileNotFoundError: [Errno 2] No such file or directory: '-f'

No module named 'tensorflow.contrib'

Which tensorflow version should I install to fix this? 'tensorflow.contrib' doesn't exist in 2.0

Unable to process data from flickr_logos_27_dataset\flickr_logos_27_dataset_cropped_augmented_images\Adidas\train.pickle : could not broadcast input array from shape (4412,32,64,3) into shape (35000,32,64,3)

Hello, I am new to tensorflow so I tried to reduce the number of logos to only one (Adidas) to make it run faster.
Removed everything except Adidas data from flickr_logos_27_dataset_query_set_annotation.txt and flickr_logos_27_dataset_training_set_annotation.txt.
Edited common.py:

CLASS_NAME = [
    'Adidas', 'Background'
]

train and test folders are created for both Adidas and Background.
So when I try to run gen_train_valid_test.py, it shows and error:

Unable to process data from flickr_logos_27_dataset\flickr_logos_27_dataset_cropped_augmented_images\Adidas\train.pickle : could not broadcast input array from shape (4412,32,64,3) into shape (35000,32,64,3)
Traceback (most recent call last):
File "c:\Users\username\Documents\Workspace\DeepLogo\gen_train_valid_test.py", line 196, in
main()
File "c:\Users\username\Documents\Workspace\DeepLogo\gen_train_valid_test.py", line 182, in main
train_datasets, TRAIN_SIZE, VALID_SIZE)
File "c:\Users\username\Documents\Workspace\DeepLogo\gen_train_valid_test.py", line 132, in merge_datasets
train_dataset[start_t:end_t, :, :, :] = train_logo
ValueError: could not broadcast input array from shape (4412,32,64,3) into shape (35000,32,64,3)

Can you help me solving this issue. Thanks!

precision and recall metric

Hello~
Thanks for your wonderful job!
When I run "logo_detection.py" to test, I am wondering how can I evaluate my training and test in precision and recall metric. Could you share with us?

Unable to run the train_deep_logo_cnn file

I ran the crop_and_aug and the gen_train_valid_test files successfully, following which I executed the train_deep_logo_cnn file. I get a OOM error. Here's the trace back:
C:\Anaconda2\envs\py35\python.exe C:/Users/user/Downloads/DeepLogo/train_deep_logo_cnn.py
Training set (50000, 32, 64, 3) (50000,)
Valid set (5000, 32, 64, 3) (5000,)
Test set (5000, 32, 64, 3) (5000,)
Training set (50000, 64, 32, 3) (50000, 27)
Valid set (5000, 64, 32, 3) (5000, 27)
Test set (5000, 64, 32, 3) (5000, 27)
2017-06-16 20:36:35.350598: W c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\platform\cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE instructions, but these are available on your machine and could speed up CPU computations.
2017-06-16 20:36:35.350862: W c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\platform\cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE2 instructions, but these are available on your machine and could speed up CPU computations.
2017-06-16 20:36:35.351095: W c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\platform\cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE3 instructions, but these are available on your machine and could speed up CPU computations.
2017-06-16 20:36:35.351327: W c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\platform\cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2017-06-16 20:36:35.351562: W c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\platform\cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2017-06-16 20:36:35.351810: W c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\platform\cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2017-06-16 20:36:35.352067: W c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\platform\cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2017-06-16 20:36:35.352329: W c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\platform\cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
2017-06-16 20:36:36.141969: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\gpu\gpu_device.cc:940] Found device 0 with properties:
name: GeForce GTX 965M
major: 5 minor: 2 memoryClockRate (GHz) 1.15
pciBusID 0000:01:00.0
Total memory: 4.00GiB
Free memory: 3.33GiB
2017-06-16 20:36:36.142257: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\gpu\gpu_device.cc:961] DMA: 0
2017-06-16 20:36:36.142378: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\gpu\gpu_device.cc:971] 0: Y
2017-06-16 20:36:36.143175: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\gpu\gpu_device.cc:1030] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 965M, pci bus id: 0000:01:00.0)
initialized
Minibatch loss at step 0: 66399.546875
Minibatch accuracy: 0.0%
2017-06-16 20:36:52.850640: W c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:273] Allocator (GPU_0_bfc) ran out of memory trying to allocate 117.19MiB. Current allocation summary follows.
2017-06-16 20:36:52.851405: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:643] Bin (256): Total Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.
2017-06-16 20:36:52.852243: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:643] Bin (512): Total Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.
2017-06-16 20:36:52.853164: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:643] Bin (1024): Total Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.
2017-06-16 20:36:52.854093: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:643] Bin (2048): Total Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.
2017-06-16 20:36:52.855036: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:643] Bin (4096): Total Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.
2017-06-16 20:36:52.855967: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:643] Bin (8192): Total Chunks: 1, Chunks in use: 0 11.8KiB allocated for chunks. 7.0KiB client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.
2017-06-16 20:36:52.856908: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:643] Bin (16384): Total Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.
2017-06-16 20:36:52.857840: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:643] Bin (32768): Total Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.
2017-06-16 20:36:52.858762: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:643] Bin (65536): Total Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.
2017-06-16 20:36:52.859684: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:643] Bin (131072): Total Chunks: 1, Chunks in use: 0 216.0KiB allocated for chunks. 4B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.
2017-06-16 20:36:52.860662: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:643] Bin (262144): Total Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.
2017-06-16 20:36:52.861635: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:643] Bin (524288): Total Chunks: 1, Chunks in use: 0 800.0KiB allocated for chunks. 300.0KiB client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.
2017-06-16 20:36:52.862653: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:643] Bin (1048576): Total Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.
2017-06-16 20:36:52.869983: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:643] Bin (2097152): Total Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.
2017-06-16 20:36:52.870734: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:643] Bin (4194304): Total Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.
2017-06-16 20:36:52.871491: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:643] Bin (8388608): Total Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.
2017-06-16 20:36:52.872356: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:643] Bin (16777216): Total Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.
2017-06-16 20:36:52.873183: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:643] Bin (33554432): Total Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.
2017-06-16 20:36:52.873948: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:643] Bin (67108864): Total Chunks: 1, Chunks in use: 0 64.00MiB allocated for chunks. 1.50MiB client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.
2017-06-16 20:36:52.874735: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:643] Bin (134217728): Total Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.
2017-06-16 20:36:52.875504: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:643] Bin (268435456): Total Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.
2017-06-16 20:36:52.876278: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:660] Bin for 117.19MiB was 64.00MiB, Chunk State:
2017-06-16 20:36:52.878767: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:666] Size: 64.00MiB | Requested Size: 1.50MiB | in_use: 0, prev: Size: 512B | Requested Size: 512B | in_use: 1, next: Size: 8.0KiB | Requested Size: 8.0KiB | in_use: 1
2017-06-16 20:36:52.879971: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000702500000 of size 1280
2017-06-16 20:36:52.880760: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000702500500 of size 256
2017-06-16 20:36:52.881668: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000702500600 of size 256
2017-06-16 20:36:52.882104: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000702500700 of size 256
2017-06-16 20:36:52.882527: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000702500800 of size 256
2017-06-16 20:36:52.883397: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000702500900 of size 512
2017-06-16 20:36:52.884240: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000702500B00 of size 8192
2017-06-16 20:36:52.884919: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000702502B00 of size 256
2017-06-16 20:36:52.885345: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000702502C00 of size 256
2017-06-16 20:36:52.885763: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000702502D00 of size 256
2017-06-16 20:36:52.886198: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000702502E00 of size 256
2017-06-16 20:36:52.886678: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000702502F00 of size 14592
2017-06-16 20:36:52.887172: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000702506800 of size 256
2017-06-16 20:36:52.887581: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000702506900 of size 307200
2017-06-16 20:36:52.887834: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000702551900 of size 256
2017-06-16 20:36:52.888084: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000702551A00 of size 819200
2017-06-16 20:36:52.888343: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000702619A00 of size 512
2017-06-16 20:36:52.888595: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000702619C00 of size 67108864
2017-06-16 20:36:52.888861: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000706619C00 of size 8192
2017-06-16 20:36:52.889117: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 000000070661BC00 of size 221184
2017-06-16 20:36:52.889377: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000706651C00 of size 256
2017-06-16 20:36:52.889639: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000706651D00 of size 256
2017-06-16 20:36:52.889901: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000706651E00 of size 256
2017-06-16 20:36:52.890168: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000706651F00 of size 256
2017-06-16 20:36:52.890444: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000706652000 of size 256
2017-06-16 20:36:52.890716: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000706652100 of size 256
2017-06-16 20:36:52.892403: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000706652200 of size 256
2017-06-16 20:36:52.892829: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000706652300 of size 256
2017-06-16 20:36:52.893166: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000706652400 of size 256
2017-06-16 20:36:52.893383: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000706652500 of size 256
2017-06-16 20:36:52.893601: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000706652600 of size 256
2017-06-16 20:36:52.893823: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000706655600 of size 256
2017-06-16 20:36:52.894073: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000706655700 of size 307200
2017-06-16 20:36:52.894333: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 00000007066A0700 of size 256
2017-06-16 20:36:52.894588: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000706768800 of size 512
2017-06-16 20:36:52.894847: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 000000070A768A00 of size 8192
2017-06-16 20:36:52.895110: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 000000070A7A0A00 of size 256
2017-06-16 20:36:52.895377: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 000000070A7A0B00 of size 256
2017-06-16 20:36:52.895637: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 000000070A7A0C00 of size 256
2017-06-16 20:36:52.895899: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 000000070A7A0D00 of size 14592
2017-06-16 20:36:52.896167: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 000000070A7A4600 of size 14592
2017-06-16 20:36:52.896436: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 000000070A7A7F00 of size 256
2017-06-16 20:36:52.896710: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 000000070A7A8000 of size 256
2017-06-16 20:36:52.896978: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 000000070A7A8100 of size 307200
2017-06-16 20:36:52.897252: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 000000070A7F3100 of size 307200
2017-06-16 20:36:52.897537: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 000000070A83E100 of size 256
2017-06-16 20:36:52.897822: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 000000070A83E200 of size 256
2017-06-16 20:36:52.898099: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 000000070A83E300 of size 819200
2017-06-16 20:36:52.898389: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 000000070A906300 of size 819200
2017-06-16 20:36:52.898674: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 000000070A9CE300 of size 512
2017-06-16 20:36:52.898954: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 000000070A9CE500 of size 512
2017-06-16 20:36:52.901430: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 000000070A9CE700 of size 67108864
2017-06-16 20:36:52.901871: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 000000070E9CE700 of size 67108864
2017-06-16 20:36:52.902092: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 00000007129CE700 of size 8192
2017-06-16 20:36:52.902307: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 00000007129D0700 of size 8192
2017-06-16 20:36:52.902531: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 00000007129D2700 of size 221184
2017-06-16 20:36:52.902764: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000712A08700 of size 221184
2017-06-16 20:36:52.903454: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000712A3E700 of size 256
2017-06-16 20:36:52.903781: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000712A3E800 of size 256
2017-06-16 20:36:52.903996: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000712A3E900 of size 14592
2017-06-16 20:36:52.904212: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000712A42200 of size 307200
2017-06-16 20:36:52.904669: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000712A8D200 of size 819200
2017-06-16 20:36:52.904891: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000712B55200 of size 67108864
2017-06-16 20:36:52.905113: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 0000000716B55200 of size 122880000
2017-06-16 20:36:52.905345: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:678] Chunk at 000000071E085200 of size 2802052608
2017-06-16 20:36:52.905606: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:687] Free at 0000000706652700 of size 12032
2017-06-16 20:36:52.905860: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:687] Free at 00000007066A0800 of size 819200
2017-06-16 20:36:52.906121: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:687] Free at 0000000706768A00 of size 67108864
2017-06-16 20:36:52.906388: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:687] Free at 000000070A76AA00 of size 221184
2017-06-16 20:36:52.907314: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:693] Summary of in-use Chunks by size:
2017-06-16 20:36:52.907740: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:696] 32 Chunks of size 256 totalling 8.0KiB
2017-06-16 20:36:52.908070: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:696] 5 Chunks of size 512 totalling 2.5KiB
2017-06-16 20:36:52.908289: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:696] 1 Chunks of size 1280 totalling 1.3KiB
2017-06-16 20:36:52.908626: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:696] 5 Chunks of size 8192 totalling 40.0KiB
2017-06-16 20:36:52.908844: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:696] 4 Chunks of size 14592 totalling 57.0KiB
2017-06-16 20:36:52.909065: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:696] 3 Chunks of size 221184 totalling 648.0KiB
2017-06-16 20:36:52.909296: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:696] 5 Chunks of size 307200 totalling 1.46MiB
2017-06-16 20:36:52.909880: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:696] 4 Chunks of size 819200 totalling 3.13MiB
2017-06-16 20:36:52.910264: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:696] 4 Chunks of size 67108864 totalling 256.00MiB
2017-06-16 20:36:52.910488: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:696] 1 Chunks of size 122880000 totalling 117.19MiB
2017-06-16 20:36:52.910710: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:696] 1 Chunks of size 2802052608 totalling 2.61GiB
2017-06-16 20:36:52.910935: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:700] Sum Total of in-use chunks: 2.98GiB
2017-06-16 20:36:52.911180: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:702] Stats:
Limit: 3267117056
InUse: 3198955776
MaxInUse: 3198955776
NumAllocs: 267
MaxAllocSize: 2802052608

2017-06-16 20:36:52.911627: W c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\bfc_allocator.cc:277] ***_***********************************************************************xxxxxxxxxxxxxxxxxxxxxxxxx
2017-06-16 20:36:52.911992: W c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\framework\op_kernel.cc:1158] Resource exhausted: OOM when allocating tensor with shape[5000,3,64,32]
Traceback (most recent call last):
File "C:\Anaconda2\envs\py35\lib\site-packages\tensorflow\python\client\session.py", line 1139, in _do_call
return fn(*args)
File "C:\Anaconda2\envs\py35\lib\site-packages\tensorflow\python\client\session.py", line 1121, in _run_fn
status, run_metadata)
File "C:\Anaconda2\envs\py35\lib\contextlib.py", line 66, in exit
next(self.gen)
File "C:\Anaconda2\envs\py35\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 466, in raise_exception_on_not_ok_status
pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors_impl.ResourceExhaustedError: OOM when allocating tensor with shape[5000,3,64,32]
[[Node: Conv2D_3 = Conv2D[T=DT_FLOAT, data_format="NHWC", padding="SAME", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true, _device="/job:localhost/replica:0/task:0/gpu:0"](Const_5, Variable/read)]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:/Users/user/Downloads/DeepLogo/train_deep_logo_cnn.py", line 256, in
main()
File "C:/Users/user/Downloads/DeepLogo/train_deep_logo_cnn.py", line 237, in main
accuracy(valid_prediction.eval(), valid_labels))
File "C:\Anaconda2\envs\py35\lib\site-packages\tensorflow\python\framework\ops.py", line 606, in eval
return _eval_using_default_session(self, feed_dict, self.graph, session)
File "C:\Anaconda2\envs\py35\lib\site-packages\tensorflow\python\framework\ops.py", line 3928, in _eval_using_default_session
return session.run(tensors, feed_dict)
File "C:\Anaconda2\envs\py35\lib\site-packages\tensorflow\python\client\session.py", line 789, in run
run_metadata_ptr)
File "C:\Anaconda2\envs\py35\lib\site-packages\tensorflow\python\client\session.py", line 997, in _run
feed_dict_string, options, run_metadata)
File "C:\Anaconda2\envs\py35\lib\site-packages\tensorflow\python\client\session.py", line 1132, in _do_run
target_list, options, run_metadata)
File "C:\Anaconda2\envs\py35\lib\site-packages\tensorflow\python\client\session.py", line 1152, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.ResourceExhaustedError: OOM when allocating tensor with shape[5000,3,64,32]
[[Node: Conv2D_3 = Conv2D[T=DT_FLOAT, data_format="NHWC", padding="SAME", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true, _device="/job:localhost/replica:0/task:0/gpu:0"](Const_5, Variable/read)]]

Caused by op 'Conv2D_3', defined at:
File "C:/Users/user/Downloads/DeepLogo/train_deep_logo_cnn.py", line 256, in
main()
File "C:/Users/user/Downloads/DeepLogo/train_deep_logo_cnn.py", line 197, in main
w_conv3, b_conv3, w_fc1, b_fc1, w_fc2, b_fc2))
File "C:/Users/user/Downloads/DeepLogo/train_deep_logo_cnn.py", line 69, in model
tf.nn.conv2d(data, w_conv1, [1, 1, 1, 1], padding='SAME') + b_conv1)
File "C:\Anaconda2\envs\py35\lib\site-packages\tensorflow\python\ops\gen_nn_ops.py", line 399, in conv2d
data_format=data_format, name=name)
File "C:\Anaconda2\envs\py35\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 767, in apply_op
op_def=op_def)
File "C:\Anaconda2\envs\py35\lib\site-packages\tensorflow\python\framework\ops.py", line 2506, in create_op
original_op=self._default_original_op, op_def=op_def)
File "C:\Anaconda2\envs\py35\lib\site-packages\tensorflow\python\framework\ops.py", line 1269, in init
self._traceback = _extract_stack()

ResourceExhaustedError (see above for traceback): OOM when allocating tensor with shape[5000,3,64,32]
[[Node: Conv2D_3 = Conv2D[T=DT_FLOAT, data_format="NHWC", padding="SAME", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true, _device="/job:localhost/replica:0/task:0/gpu:0"](Const_5, Variable/read)]]

Process finished with exit code 1

I tried changing the number of channels in both the gen_train_valid_test, but I get the following error in the gen_train_valid_test file:
ValueError: could not broadcast input array from shape (185,32,64,3) into shape (185,32,64,5)

Where am I going wrong?

in training phase

how to solve this issue

EOFError: Ran out of input

When I run python gen_train_valid_test.py

I get the attached error.

Can you please help.?

Prediction gives wrong

Hi

I have trained flickr_logos_27_dataset, I understood this dataset contains 1000 plus images of 27 classes of logos, I have croped and augmented these images then after I have trained with specified width and height. After I got the model while testing test_deep_logo_cnn.py with downloaded images from internet its giving wrong prediction

Can anyone help me please.....

recognition for new data

Hi, I use your code to detect logo for new pic, but the result is very poor, And for the origin flickr logos the result is also poor. I'm very puzzled with this result. Could you explain why？

Regarding the Testing of the model

Hi,
The model you have trained with less Data is awesome and good.It works completely perfect .
But a small issue had occurred while testing the model.

The problem is while testing the model with test folder images it works good,But the images which are taken from outside the test folder(i.e apple logo,coca cola logo and Vodafone logo etc., which was taken from the internet )while testing showing wrong .

so what might be problem ,and suggest me the the way to solve the problem.

tensorflow.python.framework.errors_impl.NotFoundError: ssd_inception_v2.config; No such file or directory

Where can I find ssd_inception_v2.config, please help.