lxztju / pytorch_classification Goto Github PK
View Code? Open in Web Editor NEW利用pytorch实现图像分类的一个完整的代码,训练,预测,TTA,模型融合,模型部署,cnn提取特征,svm或者随机森林等进行分类,模型蒸馏,一个完整的代码
License: MIT License
利用pytorch实现图像分类的一个完整的代码,训练,预测,TTA,模型融合,模型部署,cnn提取特征,svm或者随机森林等进行分类,模型蒸馏,一个完整的代码
License: MIT License
Traceback (most recent call last):
File "./tools/train_val.py", line 176, in
train(rank, local_rank, device, args)
File "./tools/train_val.py", line 67, in train
model = ClsModel(args.model_name, args.num_classes, args.is_pretrained)
File "/root/gpf/pytorch_classification/cls_models/model.py", line 31, in init
self.base_model = getattr(torchvision.models, self.model_name)(weights=ModelWeights[self.model_name])
File "/root/miniconda3/envs/pytorch_clas/lib/python3.7/site-packages/torchvision/models/resnet.py", line 277, in resnet18
**kwargs)
File "/root/miniconda3/envs/pytorch_clas/lib/python3.7/site-packages/torchvision/models/resnet.py", line 260, in _resnet
model = ResNet(block, layers, **kwargs)
TypeError: init() got an unexpected keyword argument 'weights'
使用的包版本:
pytorch==1.8.1+cu101
torchvision==0.9.1+cu101
为什么会报上面的错误?
训练集包括测试集的图片的大小都要一样吗?
...... Initialize the network done!!! .......
Traceback (most recent call last):
File "/home/jiannan/project/pytorch_classification-master/train.py", line 76, in
optimizer = optim.Adam(filter(lambda p: p.requires_grad, model.parameters()), lr=cfg.LR)
File "/home/jiannan/anaconda3/envs/pt-gpu/lib/python3.6/site-packages/torch/optim/adam.py", line 42, in init
super(Adam, self).init(params, defaults)
File "/home/jiannan/anaconda3/envs/pt-gpu/lib/python3.6/site-packages/torch/optim/optimizer.py", line 46, in init
raise ValueError("optimizer got an empty parameter list")
ValueError: optimizer got an empty parameter list
这个问题要怎么解决???
老铁,有数据集吗?求分享
训练自己的图像会出现下面的问题:
UserWarning: Corrupt EXIF data. Expecting to read 12 bytes but only got 10.
另外,在train_kd.py中,定义优化器与损失函数那里,model是标红的,直接运行,显示没有定义;
当用teacher_model,student_model代替model时,显示下面这个bug。
optimizer = optim.Adam(filter(lambda p: p.requires_grad, model.parameters()), lr=cfg.LR)
optimizer = optim.SGD(model.parameters(), lr=cfg.LR,
momentum=cfg.MOMENTUM, weight_decay=cfg.WEIGHT_DECAY)
抱拳
您好,我的图片全部在一个文件夹里,对应有一个csv文件,里面包含图片的名字和标签,这样如何转化为该项目数据的标准
在运行 train.py 时出现了这个问题,提示错误在train.py的第120行batch_iterator = iter(train_dataloader)这句话,请问如何解决呢?
from data import get_train_transform, get_test_transform
from data import get_random_eraser
import numpy as np
def get_random_eraser(p=0.5, s_l=0.02, s_h=0.4, r_1=0.3, r_2=3.3333333333333335, v_l=0, v_h=255, pixel_level=False):
def eraser(input_img):
img_h, img_w, img_c = input_img.shape
p_1 = np.random.rand()
if p_1 > p:
return input_img
else:
while 1:
s = np.random.uniform(s_l, s_h) * img_h * img_w
r = np.random.uniform(r_1, r_2)
w = int(np.sqrt(s / r))
h = int(np.sqrt(s * r))
left = np.random.randint(0, img_w)
top = np.random.randint(0, img_h)
if left + w <= img_w:
if top + h <= img_h:
break
if pixel_level:
c = np.random.uniform(v_l, v_h, (h, w, img_c))
else:
c = np.random.uniform(v_l, v_h)
input_img[top:top + h, left:left + w, :] = c
return input_img
return eraser
我看这里的预测程序里面没有score,
运行 train.py 文件时提示如下错误
No such file or directory: '/home/dll/weights/resnext101_32x32.pth'
Hi:
在使用您代码的过程中,我对比了nn.CrossEntropyLoss()和LabelSmoothingCrossEntropy(),并在每个step打印loss值,得到以下,左边为smoothing loss,右边为nn.CrossEntropyLoss,似乎smoothing的loss稳定且只能以微小变化(0.5~0.6),而右边则还能降得更低。
我在自己写的代码上同样遇到过,请问您有没留意到这个问题?期待您的回复
0.6104838848114014 0.13801012933254242
0.609204888343811 0.14818525314331055
0.6149349212646484 0.1491081565618515
0.6321774125099182 0.17487832903862
0.6126755475997925 0.1317736655473709
0.6100113987922668 0.1340378224849701
0.6236982345581055 0.15228210389614105
0.5979918837547302 0.11493214219808578
0.6044023036956787 0.12213070690631866
0.5954407453536987 0.1088051125407219
Epoch:3 || epochiter: 32/139|| Totel iter 310 || Loss: 0.595441||ACC: 99.219 ||LR: 0.00044660
0.6146756410598755 0.13752762973308563
0.6278829574584961 0.1602025330066681
0.6053824424743652 0.1260662078857422
0.6094704270362854 0.14142774045467377
0.6054157018661499 0.13225990533828735
0.6109800934791565 0.14464853703975677
0.6221442818641663 0.14965230226516724
0.6030154228210449 0.12700901925563812
0.6105899214744568 0.13878554105758667
0.6118856072425842 0.13709238171577454
Epoch:3 || epochiter: 42/139|| Totel iter 320 || Loss: 0.611886||ACC: 99.219 ||LR: 0.00046097
0.6141043305397034 0.15106570720672607
0.6156240701675415 0.16219794750213623
0.6021517515182495 0.1331876516342163
0.5985877513885498 0.12620455026626587
0.6136577129364014 0.1546640545129776
0.6055587530136108 0.12843844294548035
0.6039034128189087 0.13453435897827148
0.600081741809845 0.12076513469219208
0.5962940454483032 0.11982955038547516
0.6148940324783325 0.13918887078762054
AttributeError: 'VGG' object has no attribute 'fc'
在data文件夹里面 from . import random_eraser 没有找到random_eraser模块
请教大佬,我在导入自己图像(均为png格式)之后调用resnet101尝试训练时报错,我把train.py里的第156行labels.处加了long()之后就能够正常运行,这样改的话会不会对最终结果产生比较严重的影响?还请点拨一二
我使用resnet101网络模型,train和predict都可以正常运行,但运行cnn_ml出错了,大佬可以解答下吗?
在训练train.py 时,需要加上 loss.requires_grad_(True) 不然会报错
RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn
请问作者做分类,有遇到这个问题吗?
****** using single gpu to training ********
...... Initialize the network done!!! .......
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\ProgramData\Anaconda3\lib\multiprocessing\spawn.py", line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
File "C:\ProgramData\Anaconda3\lib\multiprocessing\spawn.py", line 125, in _main
prepare(preparation_data)
File "C:\ProgramData\Anaconda3\lib\multiprocessing\spawn.py", line 236, in prepare
_fixup_main_from_path(data['init_main_from_path'])
File "C:\ProgramData\Anaconda3\lib\multiprocessing\spawn.py", line 287, in _fixup_main_from_path
main_content = runpy.run_path(main_path,
File "C:\ProgramData\Anaconda3\lib\runpy.py", line 265, in run_path
return _run_module_code(code, init_globals, run_name,
File "C:\ProgramData\Anaconda3\lib\runpy.py", line 97, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "C:\ProgramData\Anaconda3\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "C:\Users\yihen\Downloads\pytorch_classification-master\pytorch_classification-master\train.py", line 111, in <module>
batch_iterator = iter(train_dataloader)
File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 359, in __iter__
return self._get_iterator()
File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 305, in _get_iterator
return _MultiProcessingDataLoaderIter(self)
File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 918, in __init__
w.start()
File "C:\ProgramData\Anaconda3\lib\multiprocessing\process.py", line 121, in start
self._popen = self._Popen(self)
File "C:\ProgramData\Anaconda3\lib\multiprocessing\context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "C:\ProgramData\Anaconda3\lib\multiprocessing\context.py", line 327, in _Popen
return Popen(process_obj)
File "C:\ProgramData\Anaconda3\lib\multiprocessing\popen_spawn_win32.py", line 45, in __init__
prep_data = spawn.get_preparation_data(process_obj._name)
File "C:\ProgramData\Anaconda3\lib\multiprocessing\spawn.py", line 154, in get_preparation_data
_check_not_importing_main()
File "C:\ProgramData\Anaconda3\lib\multiprocessing\spawn.py", line 134, in _check_not_importing_main
raise RuntimeError('''
RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.
This probably means that you are not using fork to start your
child processes and you have forgotten to use the proper idiom
in the main module:
if __name__ == '__main__':
freeze_support()
...
The "freeze_support()" line can be omitted if the program
is not going to be frozen to produce an executable.
after running the train.py, on my own data set, it gives this error.
作者您好,我在使用您的代码训练自己的数据时,数据一共有6类,预测结果没有问题,准确率很高,但是其他的类得分为什么会出现负数呢? 并且不同的图像得分的大小程度也差很多,请问是什么原因呢
你好,作为刚入门的选手,很开心能看到这么好的代码库。我有一个问题:我发现代码库里都是在调用现成的网络的预训练模型,能否加入我自己修改的网络训练呢?
您好,想请教一下,LOCAL_PRETRAINED这部分的功能实现代码具体是在哪个位置,好像除了在build_model.py中看到from models import LOCAL_PRETRAINED, model_urls以外,就没有看到功能具体实现的代码了。
比如,我应该把下载下来的预训练模型存储在那个位置?
是什么的特征值?怎么求到这个2816的?
运行train.py的时候报错ValueError: optimizer got an empty parameter list,报错位置optimizer = optim.Adam(filter(lambda p: p.requires_grad, model.parameters()), lr=cfg.LR)这个是什么问题呀
torch1.4.0 torchvision0.5.0
当运行train.py的时候,没有报错但是程序运行一半结束了
****** Training resnet101 ******
****** loading the Imagenet pretrained weights ******
****** using single gpu to training ********
...... Initialize the network done!!! .......
Process finished with exit code 0
程序输出到这里就结束了,可以指点一下吗
当自己尝试使用resnet50或者moblienetv2时都会报如下错误:
Traceback (most recent call last): File "train.py", line 147, in <module> images, labels = next(batch_iterator) File "/home/xuhaowang/anaconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 363, in __next__ data = self._next_data() File "/home/xuhaowang/anaconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 403, in _next_data data = self._dataset_fetcher.fetch(index) # may raise StopIteration File "/home/xuhaowang/anaconda3/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/xuhaowang/anaconda3/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp> data = [self.dataset[idx] for idx in possibly_batched_index] File "/data/xuhaow/wordload/pytorch_classification-master/data/dataset.py", line 49, in __getitem__ img_path, label = self.imgs[index] ValueError: too many values to unpack (expected 2)
请问这种情况有人遇到过么,该如何解决
你好 请问您有微信或者qq吗 我有些问题想直接问您。我现在在日本大学的研究课题和您做的这个比较有关 所以想具体问一下。
你好,我想请问一下哪个文件的代码可以实现多模型融合预测呢?
我用3个12G的卡训练,但是第一个卡显存会爆掉,但是第二和第三张卡还有很多显存,只有设置batchsize为1才不会报错,但是用单卡来跑的话batchsize可以设置为6,请问这个问题怎么解决可以用多卡且可以用较大的batchsize
该 是不是并不支持 resume 继续训练?
RuntimeError: “nll_loss_forward_reduce_cuda_kernel_2d_index“ not implemented for ‘Int‘
There is no adjust_ learning_ rate_ step.py in the utils folder file,and hope to provide it, thanks!
can u support me the pre-train model???
在加载完模型之后出现错误ValueError: optimizer got an empty parameter list
定位到的报错位置是
File "train.py", line 76, in
optimizer = optim.Adam(filter(lambda p: p.requires_grad, model.parameters()), lr=cfg.LR)
请问这个有什么推荐的数据集用来做分类吗
大佬,可以教学一下多批次推理服务吗,谢谢
cnn_svm中的 get_test_transform 并未定义,是不是写错了?
This error occurs when I use predict.py.Strangely,this is not occurring in Ubuntu,but occurring in Win 10.
我运行train.py文件时,碰到这个错误,有人解决过吗?
你好,想问一下,出现ModuleNotFoundError: No module named 'cfg'需要怎么解决呀?谢谢
您好!
# classify the input image and then initialize the list
# of predictions to return to the client
out = model(img)
# print(out)
pred_label = torch.max(out, 1)[1].item()
# print(pred_label)
data["predictions"] = []
data["predictions"].append(label_id_name_dict[str(pred_label)])
# indicate that the request was a success
data["success"] = True
# print(data["success"])
# return the data dictionary as a JSON response
return jsonify(data)
您在这一段代码里完成了对out进行数据处理并返回json的操作,我使用的是mmdetection,并参照官方样例:
results = inference_detector(model, img)
print (results)
show_result(img, results, model.CLASSES,show=False,out_file= now + '_result.jpg')
我仔细查看了result的数据,但是我仍然不清楚该如何把result中的bbox category score信息存成dict,并返回json,希望得到您的帮助!
训练模型转文件类型提示ModuleNotFoundError: No module named 'models'
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.