Comments (12)
Sure. Everything else being the same, this is the way each image is processed and evaluated. The only difference I see is read/write of the image and GT that have been done using PIL to exactly preserve the range and the RGB channel order. Let me know if you find anything strange here.
img = np.zeros((513,513,3));
img_temp = np.asarray(Image.open(os.path.join(im_path,i[:-1]+'.jpg')),dtype=np.float32)[:,:,::-1]
img_original = img_temp
img_temp -= vgg_mean
img[:img_temp.shape[0],:img_temp.shape[1],:] = img_temp
gt = np.asarray(Image.open(os.path.join(gt_path,i[:-1]+'.png')),dtype=np.uint8)
output_list = model(Variable(torch.from_numpy(img[np.newaxis, :].transpose(0,3,1,2)).float(),volatile = True).cuda())
interp = nn.UpsamplingBilinear2d(size=(513, 513))
output = interp(output_list[3]).cpu().data[0].numpy()
output = output[:,:img_temp.shape[0],:img_temp.shape[1]]
output = output.transpose(1,2,0)
output = np.argmax(output,axis = 2)
hist += fast_hist(gt.flatten(),output.flatten(),num_classes)
from pytorch-deeplab-resnet.
Yes, I found out about the mistake in the eval script just recently and I will be fixing it soon. Thank you for pointing this out. The mIoU of 72.1% could be due to issue #4.(will also be fixed soon). There is also one more difference between the caffe version and the pytorch version(during training)- the caffe version does scaling(0.5,0.75,1,1.25,1.5) on some fixed scales, while the pytorch version randomly picks a number between (0.5, 1.3). Randomly picking a number between (0.5, 1.5) does not fit while training on Titan X.
from pytorch-deeplab-resnet.
Is this bug fixed?
from pytorch-deeplab-resnet.
The fixed evaluation script is in the dev branch. Find the path of this new script in the Results sections of Readme.
from pytorch-deeplab-resnet.
The evaluation script has been fixed now. evalpyt2.py
is the correct script. The old script, evalpyt.py
, is still there to maintain continuity, and the difference between them has been clearly mentioned in the results section of readme . We get 71.13% mean IOU from the pytorch trained model. train_iter_20000.caffemodel
gives 74.39%. The converted .pth model also gives 74.39%. Readme has also been updated to provide scripts to verify each of these performance claim. Please note that in the ground truth images, the label 255 is merged with background during the evaluation because this was done during training also.
from pytorch-deeplab-resnet.
@swamiviv you said(and also in table 4 of the paper) that the .caffemodel gives 76.3% on the val set, but I am getting 74.39% only. Why could this be, is this because I am merging the boundary(255) label with the background? Are you able to get 76.3% yourself? I am using this script. In the end, I print 2 values, the second one is the one which should be considered.
from pytorch-deeplab-resnet.
from pytorch-deeplab-resnet.
Update: after leaving the 255 label as it is(as suggested by you), I am now getting 75.54% from the train_iter_20000.caffemodel
which is still lower than that reported in the paper(76.3%). If you could look into my code to find the possible cause of this, it would be great.
from pytorch-deeplab-resnet.
from pytorch-deeplab-resnet.
If you are short on time, may be you could share the script with me and I could look for differences?
from pytorch-deeplab-resnet.
even after using the above code, I get the exact same result as I got before (75.54% mean IOU)! Why could this be happening?
from pytorch-deeplab-resnet.
@isht7, hi, have you solved the problem? Did you get 76.35% as reported in the paper? By the way, how can I convert the gt images and the output images to color images? Or How did you process the gt images at the beginning?
from pytorch-deeplab-resnet.
Related Issues (20)
- Why is loss for multiple scales calculated in a strange way? HOT 2
- Arbitrary batch size still unimplemented?
- Training produces model generating blank segmentations HOT 6
- How do get car mask only? HOT 1
- how does iter_size work? HOT 1
- spatial sizes mismatch HOT 1
- the result? HOT 4
- Why change the image channel order after `cv2.imread`
- No Relu in the ClassifierModule HOT 1
- Frozen the statistics of BN? HOT 5
- The size of the prediction HOT 2
- Bad mIOU tested with provided model HOT 8
- problem of train.py HOT 1
- ASPP or LargeFOV? Should be 76.35%.
- docopt HOT 2
- network stucture issue
- read the ground truth of the pascal voc dataset
- where is the crfs implementation? HOT 1
- About image preprocessing HOT 2
- performance HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pytorch-deeplab-resnet.