titu1994 / neural-image-assessment Goto Github PK
View Code? Open in Web Editor NEWImplementation of NIMA: Neural Image Assessment in Keras
License: MIT License
Implementation of NIMA: Neural Image Assessment in Keras
License: MIT License
why does the model not support resizing option for evaluation?
when i run 'evaluate_mobilenet.py',there occured an error——No module named path.so i want to know what dose this module depend on.
When I run "python train_nasnet_mobile.py", It failed as follows:
1250/1250 [==============================] - 1439s 1s/step - loss: 0.0667 - val_loss: 0.0653
Exception IndexError: IndexError('list index out of range',) in <generator object val_generator at 0x24cf7c30> ignored
Exception IndexError: 'list index out of range' in <generator object get_controller at 0xd611500> ignored
Exception IndexError: IndexError('list index out of range',) in <generator object train_generator at 0x24cf7be0> ignored
Exception IndexError: 'list index out of range' in <generator object get_controller at 0xd611730> ignored
1)I didnt find anywhere a list index out of range.
2)I didnt find any file contains the Intermediate results,did you save the model that trained every epoch?
thks very much
Using the AVA dataset, we came across the following problems.
It shows that 'TypeError: Can't instantiate abstract class DatasetV1 with abstract methods _as_variant_tensor, _inputs'. We've googled it but cannot find the suitable answer.
Please help us.
Environment:
tensorflow-gpu 1.12.0
keras 2.24
python 3.6.6
Logs:
Traceback (most recent call last):
File "train_mobilenet.py", line 88, in <module>
validation_steps=(5000. // batchsize))
File "/usr/local/lib/python3.6/dist-packages/keras/legacy/interfaces.py", line 91, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/keras/engine/training.py", line 1418, in fit_generator
initial_epoch=initial_epoch)
File "/usr/local/lib/python3.6/dist-packages/keras/engine/training_generator.py", line 181, in fit_generator
generator_output = next(output_generator)
File "/usr/local/lib/python3.6/dist-packages/keras/utils/data_utils.py", line 709, in get
six.reraise(*sys.exc_info())
File "/home/hadoop/anaconda3/lib/python3.6/site-packages/six.py", line 693, in reraise
raise value
File "/usr/local/lib/python3.6/dist-packages/keras/utils/data_utils.py", line 685, in get
inputs = self.queue.get(block=True).get()
File "/home/hadoop/anaconda3/lib/python3.6/multiprocessing/pool.py", line 644, in get
raise self._value
File "/home/hadoop/anaconda3/lib/python3.6/multiprocessing/pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "/usr/local/lib/python3.6/dist-packages/keras/utils/data_utils.py", line 626, in next_sample
return six.next(_SHARED_SEQUENCES[uid])
File "/home/hadoop/Documents/demo-ava-1/neural-image-assessment-master/utils/data_loader.py", line 101, in train_generator
train_dataset = tf.data.Dataset().from_tensor_slices((train_image_paths, train_scores))
TypeError: Can't instantiate abstract class DatasetV1 with abstract methods _as_variant_tensor, _inputs
Thanks for job!
I read your codes. you train ava dataset wich mobiblenet,but I think this model design for mobile and its accuracy is not as good as inception_v3. Since this is a classification task, it can also train wich inception_v3. How do you think?
Hi,
Just asking, Does this code produce the same accuracy of the NIMA paper?
where did you get the dataset?
Hello, how long does it take for you to train with 250000 images? I am training with a small sample of 1000 images, and each epoch is taking about 1 hour. Is this time normal I haven't done much machine learning before so I don't have a good concept of this. The machine i use have a good nvidia gpu. Do i need any adjustment or tensorflow can automatically find the gpu and use it? Currently i only installed tensorflow 1.4. thanks for the help!
Suppose we have 3 samples, each y_truth and y_pred as following:
import numpy as np
import tensorflow as tf
from keras import backend as K
y_truth = np.array([
[0, 0, 0, 0, 0, 0, 0, 0.9, 0.1, 0],
[0, 0, 0, 0, 0, 0, 0, 0.9, 0.1, 0],
[0, 0, 0, 0, 0, 0, 0, 0.9, 0.1, 0]])
y_pred = np.array([
[0, 0, 0, 0, 0, 0, 0.9, 0, 0.1, 0],
[0.9, 0, 0, 0, 0, 0, 0, 0, 0.1, 0],
[0.8, 0, 0, 0, 0, 0, 0, 0, 0.1, 0.1]])
Each sample has its own EMD. Let's calculate them.
sess = tf.Session()
def emd_sample_wise(y_truth, y_pred):
return K.sqrt(K.mean(K.square(K.abs(K.cumsum(y_truth, axis=-1) - K.cumsum(y_pred, axis=-1))), axis=-1))
sess.run(emd_sample_wise(y_truth, y_pred))
## array([ 0.28460499, 0.75299402, 0.67082039])
Looks right. First sample has the smallest loss.
Now, we need the mean of them, which is (0.28460499 + 0.75299402 + 0.67082039)/3 = 0.56947313
def emd(y_truth, y_pred):
return K.mean(emd_sample_wise(y_truth, y_pred))
sess.run(emd(y_truth, y_pred))
## 0.56947313551525303
Comparing with the original version:
def earth_mover_loss(y_true, y_pred):
return K.sqrt(K.mean(K.square(K.abs(K.cumsum(y_true, axis=-1) - K.cumsum(y_pred, axis=-1)))))
sess.run(earth_mover_loss(y_truth, y_pred))
## 0.60497933849016694
That's incorrect.
hello! Just got another question.When i train the nasnet mobile model, the loss becomes nan during the fifth epoch. Any possible reason for that? Should i tune some parameters? I used some other data before and it did not happen. Thanks
Hi titu,
Sorry to bother you again , I just tried with 0.5 release and the HEAD of git with mobilenet(seems nasnet only process 224224 image? cut big image to 224224 to evaluate?) ,the test images are same but the score turn out to be quite different, a very bad image get high score in the the HEAD of git
do you got any idea on it?
outputs from 0.5 release:
1) output.jpg : Score = 3.76305 <<<<< optimized images
2) img_20170531_215051.jpg : Score = 3.69984
3) img_20170531_215157.jpg : Score = 3.67825
4) img_20170531_215130.jpg : Score = 3.66773
5) img_20170531_215126.jpg : Score = 3.65906
6) img_20170531_215106.jpg : Score = 3.65685
7) img_20170531_215153.jpg : Score = 3.65632
8) img_20170531_215143.jpg : Score = 3.65036
9) img_20170531_215139.jpg : Score = 3.64920
10) img_20170531_215112.jpg : Score = 3.64874
11) img_20170531_215117.jpg : Score = 3.64370
12) img_20170531_215134.jpg : Score = 3.64089
13) img_20170531_215147.jpg : Score = 3.62634
14) single_shoot_test3.jpg : Score = 3.22735 <<<< very bad images
and output from head of git
1) output.jpg : Score = 4.98324 <<<<< optimized images
2) single_shoot_test3.jpg : Score = 4.82322 <<<< very bad images
3) img_20170531_215139.jpg : Score = 4.82291
4) img_20170531_215134.jpg : Score = 4.81500
5) img_20170531_215051.jpg : Score = 4.78660
6) img_20170531_215153.jpg : Score = 4.77603
7) img_20170531_215130.jpg : Score = 4.76752
8) img_20170531_215117.jpg : Score = 4.75221
9) img_20170531_215147.jpg : Score = 4.73955
10) img_20170531_215126.jpg : Score = 4.72469
11) img_20170531_215106.jpg : Score = 4.71353
12) img_20170531_215157.jpg : Score = 4.71176
13) img_20170531_215143.jpg : Score = 4.69852
14) img_20170531_215112.jpg : Score = 4.68663
0.5 release modifications to read imgs in dir in my env
import numpy as np
import os
from keras.models import Model
from keras.layers import Dense, Dropout
from keras.applications.mobilenet import MobileNet
from keras.applications.mobilenet import preprocess_input
from keras.preprocessing.image import load_img, img_to_array
import tensorflow as tf
import sys
sys.path.append("./utils")
from score_utils import mean_score, std_score
#get jpg file from path
def get_imlist(path):
return [os.path.join(path,f) for f in os.listdir(path) if f.endswith('.jpg')]
def envaluation(model,img_path):
img = load_img(img_path)
x = img_to_array(img)
x = np.expand_dims(x, axis=0)
x = preprocess_input(x)
scores = model.predict(x, batch_size=1, verbose=1)[0]
mean = mean_score(scores)
std = std_score(scores)
file_name = os.path.basename(img_path).lower()
score_list.append((file_name, mean))
print("Evaluating : ", img_path)
print("NIMA Score : %0.3f +- (%0.3f)" % (mean, std))
with tf.device('/CPU:0'):
base_model = MobileNet((None, None, 3), alpha=1, include_top=False, pooling='avg', weights=None)
x = Dropout(0.75)(base_model.output)
x = Dense(10, activation='softmax')(x)
model = Model(base_model.input, x)
model.load_weights('weights/mobilenet_weights.h5')
#Get All the original Photo
imgs = get_imlist("./inputs/")
score_list = []
for img_path in imgs[1:]:
try:
envaluation(model,img_path)
except:
print (img_path + '...skipped')
print("*" * 40, "Ranking Images", "*" * 40)
score_list = sorted(score_list, key=lambda x: x[1], reverse=True)
for i, (name, score) in enumerate(score_list):
print("%d)" % (i + 1), "%s : Score = %0.5f" % (name, score))
Total params: 4,280,286
Trainable params: 10,570
Non-trainable params: 4,269,716
this is what i got, i tried different dataset but it's always like this. Is this normal?Or any cause for so many non-trainable params? Also do the training images also need to be resized to 224*224 before training? Thanks.
I run the train_model, and find some problems:
Traceback (most recent call last):
File "/home/ztt/下载/neural-image-assessment-master/train_model.py", line 196, in
main()
File "/home/ztt/下载/neural-image-assessment-master/train_model.py", line 188, in main
initial_epoch=0
File "/home/ztt/anaconda3/envs/tensorflow/lib/python3.5/site-packages/keras/legacy/interfaces.py", line 91, in wrapper
return func(*args, **kwargs)
File "/home/ztt/anaconda3/envs/tensorflow/lib/python3.5/site-packages/keras/engine/training.py", line 2145, in fit_generator
generator_output = next(output_generator)
File "/home/ztt/anaconda3/envs/tensorflow/lib/python3.5/site-packages/keras/utils/data_utils.py", line 561, in get
six.raise_from(StopIteration(e), e)
File "", line 3, in raise_from
StopIteration: [Errno 2] No such file or directory: 's:\datasets\AVA_dataset\images/950742.jpg'
I meet error when train mobilenet according to your guide. Please help me, thanks~
Traceback (most recent call last):
File "train_mobilenet.py", line 88, in
validation_steps=(5000. // batchsize))
File "/home/zw/work/NIMA/venv/local/lib/python2.7/site-packages/keras/legacy/interfaces.py", line 91, in wrapper
return func(*args, **kwargs)
File "/home/zw/work/NIMA/venv/local/lib/python2.7/site-packages/keras/engine/training.py", line 2192, in fit_generator
generator_output = next(output_generator)
File "/home/zw/work/NIMA/venv/local/lib/python2.7/site-packages/keras/utils/data_utils.py", line 793, in get
six.reraise(value.class, value, value.traceback)
File "/home/zw/work/NIMA/venv/local/lib/python2.7/site-packages/keras/utils/data_utils.py", line 658, in _data_generator_task
generator_output = next(self._generator)
File "/home/zw/work/NIMA/neural-image-assessment/utils/data_loader.py", line 102, in train_generator
train_dataset = tf.data.Dataset().from_tensor_slices((train_image_paths, train_scores))
TypeError: Can't instantiate abstract class Dataset with abstract methods _as_variant_tensor, output_shapes, output_types
The dataset print result is as below:
('Train set size : ', (250508,), (250508, 10))
('Val set size : ', (5000,), (5000, 10))
Train and validation datasets ready !
Hi ,
I tried with below inputs and get error outputs that functions in utils not found
~/code/ML/neural-image-assessment-0.5$ python3 evaluate_nasnet.py -dir ./inputs/ -resize true
Using TensorFlow backend.
Traceback (most recent call last):
File "evaluate_nasnet.py", line 8, in <module>
from utils.nasnet import NASNetMobile, preprocess_input
ImportError: No module named 'utils.nasnet'
do I need add below code to each script or I miss some settings?
import sys
sys.path.append(path To Folder Containing Scripts)
edit: using the 0.5 release version
I want to train a model on tid2013,so i changed some details in the script of data_loader.py.but i got an error:
Traceback (most recent call last):
File "train_mobilenet_tid.py", line 90, in
validation_steps=(500. // batchsize))
File "/usr/local/lib/python2.7/dist-packages/keras/legacy/interfaces.py", line 91, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 1418, in fit_generator
initial_epoch=initial_epoch)
File "/usr/local/lib/python2.7/dist-packages/keras/engine/training_generator.py", line 181, in fit_generator
generator_output = next(output_generator)
File "/usr/local/lib/python2.7/dist-packages/keras/utils/data_utils.py", line 709, in get
six.reraise(*sys.exc_info())
File "/usr/local/lib/python2.7/dist-packages/keras/utils/data_utils.py", line 685, in get
inputs = self.queue.get(block=True).get()
File "/usr/lib/python2.7/multiprocessing/pool.py", line 567, in get
raise self._value
TypeError: Can't instantiate abstract class Dataset with abstract methods _as_variant_tensor, output_classes, output_shapes, output_types
I am so confused of this error,could you help me with this?
I was able to successfully save the MobileNet NIMA model as a coreML model however the input is not what I was expecting. It shows the input as MultiArray instead of an Image
base_model = MobileNet((None, None, 3), alpha=1, include_top=False, pooling='avg', weights=None)
x = Dropout(0.75)(base_model.output)
x = Dense(10, activation='softmax')(x)
model = Model(base_model.input, x)
model.load_weights('weights/mobilenet_weights.h5')
coreml_model = coremltools.converters.keras.convert(model)
coreml_model.save("mobilenet_nima2.mlmodel")
Ideally what the input and output should resemble:
I also tried converting the model by passing in an image_input_names parameter
coreml_model = coremltools.converters.keras.convert(model, image_input_names='img', input_names='img')
but I received an unpack error from the coreML builder tools
/coremltools/models/neural_network/builder.py", line 2547, in set_pre_processing_parameters
channels, height, width = array_shape
ValueError: need more than 1 value to unpack
Code where the error originates
https://github.com/apple/coremltools/blob/master/coremltools/models/neural_network.py
# Add image inputs
for input_ in spec.description.input:
if input_.name in image_input_names:
if input_.type.WhichOneof('Type') == 'multiArrayType':
array_shape = tuple(input_.type.multiArrayType.shape)
channels, height, width = array_shape
Any idea why the keras model is converting to a CoreML model with input MultiArray type instead of an image?
There have been MobileNet models that have been successfully converted to a CoreML model with image as an input e.g. https://github.com/hollance/MobileNet-CoreML
hello!
i tried to train with vgg16 structure.
but, after epochs are done, error codes come out like this.
and the data's format is same as AVA.
Exception ignored in: <generator object val_generator at 0x7f878d041728>
Traceback (most recent call last):
File "/home/kpark2/neural-image-assessment/utils/data_loader.py", line 160, in val_generator
yield (X_batch, y_batch)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1558, in exit
self._default_graph_context_manager.exit(exec_type, exec_value, exec_tb)
File "/usr/lib/python3.5/contextlib.py", line 77, in exit
self.gen.throw(type, value, traceback)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 4973, in get_controller
context.context_stack.pop()
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/eager/context.py", line 100, in pop
self.stack.pop()
IndexError: pop from empty list
Exception ignored in: <generator object train_generator at 0x7f878d0416d0>
Traceback (most recent call last):
File "/home/kpark2/neural-image-assessment/utils/data_loader.py", line 125, in train_generator
yield (X_batch, y_batch)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1558, in exit
self._default_graph_context_manager.exit(exec_type, exec_value, exec_tb)
File "/usr/lib/python3.5/contextlib.py", line 77, in exit
self.gen.throw(type, value, traceback)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 4973, in get_controller
context.context_stack.pop()
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/eager/context.py", line 100, in pop
self.stack.pop()
IndexError: pop from empty list
help me!
please provide the tips on how to install.
I found the emd definition here https://github.com/titu1994/neural-image-assessment/blob/master/train_mobilenet.py#L49
def earth_mover_loss(y_true, y_pred):
return K.sqrt(K.mean(K.square(K.abs(y_true - y_pred))))
You are missing the CDF
function. According to the paper: EMD(y_true, y_pred) = sqrt(mean(abs(CDF(y_true) - CDF(y_pred))))
y_true = [0, 0, 0, 0, 0, 0, 0, 0.9, 0.1, 0]
y_pred1 = [0, 0, 0, 0, 0, 0, 0.9, 0, 0.1, 0]
y_pred2 = [0.9, 0, 0, 0, 0, 0, 0, 0, 0.1, 0]
y_pred1's loss should be less than y_pred2's loss
def _preprocess_symbolic_input(x, data_format, mode):
"""Preprocesses a symbolic image tensor.
Arguments:
x: symoblic tensor, 3D or 4D.
data_format: data format of the image tensor.
mode: One of "caffe", "tf".
- caffe: will convert the images from RGB to BGR,
then will zero-center each color channel with
respect to the ImageNet dataset,
without scaling.
- tf: will scale pixels between -1 and 1,
sample-wise.
Returns:
Preprocessed tensor.
"""
global _IMAGENET_MEAN
if mode == 'tf':
x /= 127.5
x -= 1.
return x
if data_format == 'channels_first':
# 'RGB'->'BGR'
if K.ndim(x) == 3:
x = x[::-1, ...]
else:
x = x[:, ::-1, ...]
else:
# 'RGB'->'BGR'
x = x[..., ::-1]
if _IMAGENET_MEAN is None:
_IMAGENET_MEAN = K.constant(-np.array([103.939, 116.779, 123.68]))
# Zero-center by mean pixel
if K.dtype(x) != K.dtype(_IMAGENET_MEAN):
x = K.bias_add(x, K.cast(_IMAGENET_MEAN, K.dtype(x)), data_format)
else:
x = K.bias_add(x, _IMAGENET_MEAN, data_format)
return x
https://github.com/titu1994/neural-image-assessment/blob/master/data_loader.py#L59
IMAGENET_MEAN = [103.939, 116.779, 123.68]
or just [127, 127, 127]
?So, I was just refactoring my fork of the code and I got bad results in the nasnet evaluation script.
It turns out there's a bug when loading the weights.
model.load_weights('weights/nasnet_weights.h5', by_name=True)
Remove by_name=true! Most layers have names generated on the fly, except maybe the fully connected layer. So what this does is probably only loading the fully connected layer's weights!
If you load the complete network, you will get good results!
Hi
could I ask which version of tf, keras and python you used to train and test all the networks please?
I had the same try with you about NIMA, but I cannot reach such high SRCC in the paper. Could you reach that? By the way, I have a different implement with you, but I agree with your loss function. Thank you!
Histograms of the ground truth and predicted scores from the article, p.7
And did the same for two models here:
It shows that MobileNet will show correct scores for very few ground truth images with score <4 and > 7.
Better one is this implementation (mobilenet): https://github.com/idealo/image-quality-assessment
Actually, I'm having troubles myself trying to fit MobileNet2, I'm getting something similar to your mobilenet image.
My histograms are bult on 0.1 subset of entire set.
Hi,
thanks for implementing this paper!
I just played a little bit with it, and I noticed almost every image, even really good or really bad ones all had scores of at least 4.8 and rarely more than 5.x.
Could this be a bug? Or maybe is more training needed? I was using your weights.
How long did you train for?
Thank you so much for your fantastic work!
I have just read you code and try to redo it again.
Then i just change the batch to 10, epochs to 2, and use the first 250 pics as training set and the last 50 as val set.
At the first, It seems to be fine.
But I get the Index error in the final epoch.
Exception ignored in: <generator object train_generator at 0x7f8c7433a360> Traceback (most recent call last): File "/home/haoyuzhang/workspace/NIMA_zhanghaoyu/utils/data_loader.py", line 123, in train_generator yield (X_batch, y_batch) File "/home/haoyuzhang/.local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1503, in __exit__ exec_type, exec_value, exec_tb) File "/usr/lib/python3.5/contextlib.py", line 77, in __exit__ self.gen.throw(type, value, traceback) File "/home/haoyuzhang/.local/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 4342, in get_controller if self.stack[-1] is not default: IndexError: list index out of range
I know there is something wrong with keras.fit_generator but I can't figure it now.
Did some one else have the same situation?
I am having trouble loading the .so(shared object) files of tensorflow during training .May i get to know the tensorflow version on which the code is executed so that i could create my environment according to that
swig/python detected a memory leak of type 'int64_t *', no destructor found.
I see that the weights besides the very top layer are frozen in train*.py
files:
base_model = InceptionResNetV2(input_shape=(image_size, image_size, 3), include_top=False, pooling='avg')
for layer in base_model.layers:
layer.trainable = False
Are the weights provided in the releases page also obtain this way?
I understand this issue is very basic problem.
In evaluate_mobilenet.py, how should I set my directry (C:\users\ ...), images, and resize option.
Could you show a sample in your case to me??
###############
parser = argparse.ArgumentParser(description='Evaluate NIMA(Inception ResNet v2)')
parser.add_argument('-dir', type=str, default=None,
help='Pass a directory to evaluate the images in it')
parser.add_argument('-img', type=str, default=[None], nargs='+',
help='Pass one or more image paths to evaluate them')
parser.add_argument('-resize', type=str, default='false',
help='Resize images to 224x224 before scoring')
parser.add_argument('-rank', type=str, default='true',
help='Whether to tank the images after they have been scored')
###############
Thank you so much ...
README.md says that @tfriedel trained weights for Inception ResNet v2, but I can't find it. Is that weights available for downloading anywhere?
test two images (one for good quality and another distorted image) from the three trained models, i got scores as below:
Evaluating_inception_resnet : D:\test\bad.PNG
NIMA Score : 5.215 +- (1.719)
Evaluating_inception_resnet : D:\test\good.PNG
NIMA Score : 5.694 +- (1.815)
Evaluating_mobilenet : D:\test\bad.PNG
NIMA Score : 5.132 +- (1.723)
Evaluating_mobilenet : D:\test\good.PNG
NIMA Score : 5.653 +- (1.651)
Evaluating_nasnet : D:\test\bad.PNG
NIMA Score : 4.879 +- (1.759)
Evaluating_nasnet : D:\test\good.PNG
NIMA Score : 6.736 +- (1.593)
it makes me confused why the distored image still got high score?
Hi:
The work is very interesting and valuable. Could you also release the VGG16 version of the NIMA in the paper. I checked the repository and only found the MobileNet , InceptionNet versions available.
Many thanks
it encouter below error when I training test data with 45 images for train_dataset and 25 images for val_dataset.
------------------- validation exception!
Exception ignored in: <generator object DataSetLoader.val_generator at 0x10eb6d888>
Traceback (most recent call last):
File "/Users/jia/IdeaProjects/lizhifm_projects/image_quality_assessment/image_assessment/data/dataset_loader.py", line 196, in val_generator
yield (x_batch, y_batch)
File "/Users/jia/.virtualenvs/tensorflow1.3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1507, in exit
exec_type, exec_value, exec_tb)
File "/Users/jia/anaconda3/lib/python3.6/contextlib.py", line 99, in exit
self.gen.throw(type, value, traceback)
File "/Users/jia/.virtualenvs/tensorflow1.3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3972, in get_controller
if self.stack[-1] is not default:
IndexError: list index out of range
------------------- train exception!
Exception ignored in: <generator object DataSetLoader.train_generator at 0x10eb6d830>
Traceback (most recent call last):
File "/Users/jia/IdeaProjects/lizhifm_projects/image_quality_assessment/image_assessment/data/dataset_loader.py", line 147, in train_generator
yield (x_batch, y_batch)
File "/Users/jia/.virtualenvs/tensorflow1.3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1507, in exit
exec_type, exec_value, exec_tb)
File "/Users/jia/anaconda3/lib/python3.6/contextlib.py", line 99, in exit
self.gen.throw(type, value, traceback)
File "/Users/jia/.virtualenvs/tensorflow1.3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3972, in get_controller
if self.stack[-1] is not default:
IndexError: list index out of range
Exception ignored in: <generator object get_controller at 0x10eb6d9e8>
Traceback (most recent call last):
File "/Users/jia/.virtualenvs/tensorflow1.3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3972, in get_controller
if self.stack[-1] is not default:
IndexError: list index out of range
Exception ignored in: <generator object get_controller at 0x112e5e360>
Traceback (most recent call last):
File "/Users/jia/.virtualenvs/tensorflow1.3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3972, in get_controller
if self.stack[-1] is not default:
IndexError: list index out of range
when I tried "python train_nasnet_mobile.py", it failed as follows:
Epoch 1/20
2018-03-22 21:01:16.558680: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1120] Creating TensorFlow device (/device:GPU:0) -> (device: 0, name: Tesla P40, pci bus id: 0000:83:00.0, compute capability: 6.1)
Exception in thread Thread-2:
Traceback (most recent call last):
File "software/python/lib/python2.7/threading.py", line 808, in __bootstrap_inner
self.run()
File "software/python/lib/python2.7/threading.py", line 761, in run
self.__target(*self.__args, **self.__kwargs)
File "software/python/lib/python2.7/site-packages/keras/utils/data_utils.py", line 630, in data_generator_task
generator_output = next(self._generator)
File "prj/NIMA/neural-image-assessment/utils/data_loader.py", line 100, in train_generator
train_dataset = tf.data.Dataset().from_tensor_slices((train_image_paths, train_scores))
TypeError: Can't instantiate abstract class Dataset with abstract methods _as_variant_tensor, output_shapes, output_types
Traceback (most recent call last):
File "train_nasnet_mobile.py", line 94, in
validation_steps=(5000. // batchsize))
File "software/python/lib/python2.7/site-packages/keras/legacy/interfaces.py", line 87, in wrapper
return func(*args, **kwargs)
File "software/python/lib/python2.7/site-packages/keras/engine/training.py", line 2083, in fit_generator
generator_output = next(output_generator)
StopIteration
Hello, i encountered the following error when using train_nasnet_mobile.py. I actually encountered the same error when using all the other training models. Evaluate models worked very well.
Total params: 4,280,286
Trainable params: 10,570
Non-trainable params: 4,269,716
Epoch 1/20
Traceback (most recent call last):
File "/Users/jiazhengyuan/venv3.62/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 510, in _apply_op_helper
preferred_dtype=default_dtype)
File "/Users/jiazhengyuan/venv3.62/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1104, in internal_convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "/Users/jiazhengyuan/venv3.62/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 947, in _TensorTensorConversionFunction
(dtype.name, t.dtype.name, str(t)))
ValueError: Tensor conversion requested dtype string for Tensor with dtype float64: 'Tensor("arg0:0", shape=(), dtype=float64)'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "train_nasnet_mobile.py", line 93, in
validation_steps=(5000. // batchsize))
File "/Users/jiazhengyuan/venv3.62/lib/python3.6/site-packages/keras/legacy/interfaces.py", line 91, in wrapper
return func(*args, **kwargs)
File "/Users/jiazhengyuan/venv3.62/lib/python3.6/site-packages/keras/engine/training.py", line 1426, in fit_generator
initial_epoch=initial_epoch)
File "/Users/jiazhengyuan/venv3.62/lib/python3.6/site-packages/keras/engine/training_generator.py", line 155, in fit_generator
generator_output = next(output_generator)
File "/Users/jiazhengyuan/venv3.62/lib/python3.6/site-packages/keras/utils/data_utils.py", line 793, in get
six.reraise(value.class, value, value.traceback)
File "/Users/jiazhengyuan/venv3.62/lib/python3.6/site-packages/six.py", line 693, in reraise
raise value
File "/Users/jiazhengyuan/venv3.62/lib/python3.6/site-packages/keras/utils/data_utils.py", line 658, in _data_generator_task
generator_output = next(self._generator)
File "/Users/jiazhengyuan/Documents/workspace/neural-image-assessment/utils/data_loader.py", line 101, in train_generator
train_dataset = train_dataset.map(parse_data, num_parallel_calls=2)
File "/Users/jiazhengyuan/venv3.62/lib/python3.6/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 853, in map
return ParallelMapDataset(self, map_func, num_parallel_calls)
File "/Users/jiazhengyuan/venv3.62/lib/python3.6/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 1870, in init
super(ParallelMapDataset, self).init(input_dataset, map_func)
File "/Users/jiazhengyuan/venv3.62/lib/python3.6/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 1839, in init
self._map_func.add_to_graph(ops.get_default_graph())
File "/Users/jiazhengyuan/venv3.62/lib/python3.6/site-packages/tensorflow/python/framework/function.py", line 484, in add_to_graph
self._create_definition_if_needed()
File "/Users/jiazhengyuan/venv3.62/lib/python3.6/site-packages/tensorflow/python/framework/function.py", line 319, in _create_definition_if_needed
self._create_definition_if_needed_impl()
File "/Users/jiazhengyuan/venv3.62/lib/python3.6/site-packages/tensorflow/python/framework/function.py", line 336, in _create_definition_if_needed_impl
outputs = self._func(*inputs)
File "/Users/jiazhengyuan/venv3.62/lib/python3.6/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 1802, in tf_map_func
ret = map_func(*nested_args)
File "/Users/jiazhengyuan/Documents/workspace/neural-image-assessment/utils/data_loader.py", line 61, in parse_data
image = tf.read_file(filename)
File "/Users/jiazhengyuan/venv3.62/lib/python3.6/site-packages/tensorflow/python/ops/gen_io_ops.py", line 528, in read_file
"ReadFile", filename=filename, name=name)
File "/Users/jiazhengyuan/venv3.62/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 533, in _apply_op_helper
(prefix, dtypes.as_dtype(input_arg.type).name))
TypeError: Input 'filename' of 'ReadFile' Op has type float64 that does not match expected type of string.
Hi @titu1994 , would you please share the AVA dataset files? I could not access the original AVA dataset page due to some reason in China mainland.
Hi titu1994,
I have just run your 3-step code and when I try to run pretrain_.py
I got this problem.
Traceback (most recent call last): File "pretrain_inception_resnet.py", line 82, in <module> model.fit_generator(features_generator(TRAIN_RECORD_PATH, batchsize=batchsize, shuffle=True), TypeError: features_generator() missing 1 required positional argument: 'faeture_size'
I knew it is because you didn't set feature in pretrain_.py
But I just can't figure out what does the faeture mean?
Hi titu, thanks for your work!
I run the training code you wrote and the generated weight file looks quite different with file posted in the release. I run a test with some images using the weight file I got and the release weight file, and find almost every image got different scores with these two weight file.
How does this happen?
Hi,
I'm using this code on AVA dataset and have encountered some problem when training. The model converges at around 10th epoch but the prediction result is not as expected. The score of all pictures are around 5.x and also the model can not figure out which picture is better. I'm using the directly training method and have tried mobilenet / vgg16 / resnet50 / inceptionv3 but none of them works.
So could you please share the exact settings for the one used in the demo images? Or can you share the well trained models ? Many thanks.
Hi, do you turn off Dropout when evaluating?
I couldn't find the related code.
i got two troubles
i achieved large training/validation loss (0.12/0.13) with VGG16 on TID2013, this result differs with nima paper released (0.054).
My experimental settings are pre-trained top layer with dropout rate 0.5, lr=1e-4, epoch=200, batch_size=50, batch=192, then fine-tuning the whole model with dropout rate 0.5, lr=3e-7, epoch=30, batch_size=50, batch=192. The dataset has 25 scenes processed by 24 distorted types with 5 levels. I use leave-one-distorted-level-out for validation, others for training.
With similar training manner, we divide dataset into three parts: training/validation/test. we use leave-one-scene-out as test, we got 25 models for each scene. But some scene predictions (10/25) are extremely bad, almost got same score with 3. i checked the image and found some failure cases are actually similar with training scene. Is this result reasonable?
Hello. You mentioned that evaluation requires cpu, train needs gpu.Just wondering would evaluation by gpu make the process faster? Thanks
Epoch 00020: val_loss did not improve from 0.04966
Exception ignored in: <generator object val_generator at 0x7f588c71da20>
Traceback (most recent call last):
File "/home/tiger/jiazheng/neural-image-assessment/utils/data_loader.py", line 158, in val_generator
yield (X_batch, y_batch)
File "/home/tiger/jiazheng/venv34/lib/python3.4/site-packages/tensorflow/python/client/session.py", line 1596, in exit
self._default_graph_context_manager.exit(exec_type, exec_value, exec_tb)
File "/usr/lib/python3.4/contextlib.py", line 77, in exit
self.gen.throw(type, value, traceback)
File "/home/tiger/jiazheng/venv34/lib/python3.4/site-packages/tensorflow/python/framework/ops.py", line 5267, in get_controller
context.context().context_switches.pop()
File "/home/tiger/jiazheng/venv34/lib/python3.4/site-packages/tensorflow/python/eager/context.py", line 136, in pop
self.stack.pop()
IndexError: pop from empty list
Exception ignored in: <generator object train_generator at 0x7f58ac2835e8>
Traceback (most recent call last):
File "/home/tiger/jiazheng/neural-image-assessment/utils/data_loader.py", line 123, in train_generator
yield (X_batch, y_batch)
File "/home/tiger/jiazheng/venv34/lib/python3.4/site-packages/tensorflow/python/client/session.py", line 1596, in exit
self._default_graph_context_manager.exit(exec_type, exec_value, exec_tb)
File "/usr/lib/python3.4/contextlib.py", line 77, in exit
self.gen.throw(type, value, traceback)
File "/home/tiger/jiazheng/venv34/lib/python3.4/site-packages/tensorflow/python/framework/ops.py", line 5267, in get_controller
context.context().context_switches.pop()
File "/home/tiger/jiazheng/venv34/lib/python3.4/site-packages/tensorflow/python/eager/context.py", line 136, in pop
self.stack.pop()
IndexError: pop from empty list
after the last epoch, this error happened. I checked the dataset, the ava.txt matches the training images. all the epochs before were good. Thanks!
Thanks for your jobs!
I download the model in 'Releases Tab' and test the predict score for the images in your README.md,but can not get the similar scores. The inception_resnet and mobilenet have the same problem. why?
I see that evaluation need to resize the pic to 224*224, is that the same for training? Thanks
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.