tencentyouturesearch / crowdcounting-p2pnet Goto Github PK

Can vgg11_bn be used instead of vgg16_bn in the code?

I attempted to change the model to the pre-trained vgg11_bn model provided by PyTorch, but I encountered an error. The error message states that some keys are missing in the state_dict for the P2PNet model, such as backbone.body1.5.weight, backbone.body1.5.bias, backbone.body1.5.running_mean, backbone.body1.5.running_var, and many others. In addition, there are unexpected keys in the state_dict, such as backbone.body1.10.weight, backbone.body1.10.bias, backbone.body1.11.weight, backbone.body1.11.bias, and so on. There is also a size mismatch for some of the parameters, such as backbone.body1.4.weight and backbone.body1.4.bias.

I found this error:

RuntimeError: Error(s) in loading state_dict for P2PNet:
        Missing key(s) in state_dict: "backbone.body1.5.weight", "backbone.body1.5.bias", "backbone.body1.5.running_mean", "backbone.body1.5.running_var", "backbone.body2.0.weight", "backbone.body2.0.bias", "backbone.body2.0.running_mean", "backbone.body2.0.running_var", "backbone.body2.3.weight", "backbone.body2.3.bias", "backbone.body2.3.running_mean", "backbone.body2.3.running_var", "backbone.body2.6.weight", "backbone.body2.6.bias", "backbone.body3.0.weight", "backbone.body3.0.bias", "backbone.body3.0.running_mean", "backbone.body3.0.running_var", "backbone.body3.3.weight", "backbone.body3.3.bias", "backbone.body3.3.running_mean", "backbone.body3.3.running_var", "backbone.body3.6.weight", "backbone.body3.6.bias", "backbone.body4.0.weight", "backbone.body4.0.bias", "backbone.body4.0.running_mean", "backbone.body4.0.running_var", "backbone.body4.3.weight", "backbone.body4.3.bias", "backbone.body4.3.running_mean", "backbone.body4.3.running_var". 
        Unexpected key(s) in state_dict: "backbone.body1.10.weight", "backbone.body1.10.bias", "backbone.body1.11.weight", "backbone.body1.11.bias", "backbone.body1.11.running_mean", "backbone.body1.11.running_var", "backbone.body1.11.num_batches_tracked", "backbone.body1.3.weight", "backbone.body1.3.bias", "backbone.body1.4.running_mean", "backbone.body1.4.running_var", "backbone.body1.4.num_batches_tracked", "backbone.body1.7.weight", "backbone.body1.7.bias", "backbone.body1.8.running_mean", "backbone.body1.8.running_var", "backbone.body1.8.num_batches_tracked", "backbone.body2.7.weight", "backbone.body2.7.bias", "backbone.body2.8.weight", "backbone.body2.8.bias", "backbone.body2.8.running_mean", "backbone.body2.8.running_var", "backbone.body2.8.num_batches_tracked", "backbone.body2.1.weight", "backbone.body2.1.bias", "backbone.body2.2.running_mean", "backbone.body2.2.running_var", "backbone.body2.2.num_batches_tracked", "backbone.body2.4.weight", "backbone.body2.4.bias", "backbone.body2.5.weight", "backbone.body2.5.bias", "backbone.body2.5.running_mean", "backbone.body2.5.running_var", "backbone.body2.5.num_batches_tracked", "backbone.body3.7.weight", "backbone.body3.7.bias", "backbone.body3.8.weight", "backbone.body3.8.bias", "backbone.body3.8.running_mean", "backbone.body3.8.running_var", "backbone.body3.8.num_batches_tracked", "backbone.body3.1.weight", "backbone.body3.1.bias", "backbone.body3.2.running_mean", "backbone.body3.2.running_var", "backbone.body3.2.num_batches_tracked", "backbone.body3.4.weight", "backbone.body3.4.bias", "backbone.body3.5.weight", "backbone.body3.5.bias", "backbone.body3.5.running_mean", "backbone.body3.5.running_var", "backbone.body3.5.num_batches_tracked", "backbone.body4.7.weight", "backbone.body4.7.bias", "backbone.body4.8.weight", "backbone.body4.8.bias", "backbone.body4.8.running_mean", "backbone.body4.8.running_var", "backbone.body4.8.num_batches_tracked", "backbone.body4.1.weight", "backbone.body4.1.bias", "backbone.body4.2.running_mean", "backbone.body4.2.running_var", "backbone.body4.2.num_batches_tracked", "backbone.body4.4.weight", "backbone.body4.4.bias", "backbone.body4.5.weight", "backbone.body4.5.bias", "backbone.body4.5.running_mean", "backbone.body4.5.running_var", "backbone.body4.5.num_batches_tracked". 
        size mismatch for backbone.body1.4.weight: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128, 64, 3, 3]).
        size mismatch for backbone.body1.4.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
        size mismatch for backbone.body1.8.weight: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256, 128, 3, 3]).
        size mismatch for backbone.body1.8.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
        size mismatch for backbone.body2.2.weight: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([256, 256, 3, 3]).
        size mismatch for backbone.body3.2.weight: copying a param with shape torch.Size([512]) from checkpoint, the shape in current model is torch.Size([512, 512, 3, 3]).
        size mismatch for backbone.body4.2.weight: copying a param with shape torch.Size([512]) from checkpoint, the shape in current model is torch.Size([512, 512, 3, 3]).

I would greatly appreciate any response. Thank you!

使用NWPU数据集训练时出现如下错误，使用ShanghaiTech PartA ,PartB训练OK
Start training
Traceback (most recent call last):
File "train.py", line 222, in
main(args)
File "train.py", line 159, in main
stat = train_one_epoch(
File "/home/model/CrowdCounting-P2PNet/engine.py", line 85, in train_one_epoch
for samples, targets in data_loader:
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/dataloader.py", line 521, in next
data = self._next_data()
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/dataloader.py", line 1183, in _next_data
return self._process_data(data)
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/dataloader.py", line 1229, in _process_data
data.reraise()
File "/usr/local/lib/python3.8/dist-packages/torch/_utils.py", line 425, in reraise
raise self.exc_type(msg)
IndexError: Caught IndexError in DataLoader worker process 6.
Original Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop
data = fetcher.fetch(index)
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/model/CrowdCounting-P2PNet/crowd_datasets/SHHA/SHHA.py", line 73, in getitem
img, point = random_crop(img, point)
File "/home/model/CrowdCounting-P2PNet/crowd_datasets/SHHA/SHHA.py", line 133, in random_crop
idx = (den[:, 0] >= start_w) & (den[:, 0] <= end_w) & (den[:, 1] >= start_h) & (den[:, 1] <= end_h)
IndexError: too many indices for array: array is 1-dimensional, but 2 were indexed

Can you provide processed shha datasets or preprocessing code? thanks!

hello ~thanks for this great job.
I get some trouble when resizing images in run_tset.py.
I modify new_width and new_height like this:

# load the images
img_raw = Image.open(data_path+img_path).convert('RGB')
# round the size
width, height = img_raw.size
img_raw = img_raw.resize((int(width*0.5), int(height*0.5)), Image.ANTIALIAS)

there is the error message：
Namespace(backbone='vgg16_bn', gpu_id=0, line=2, output_dir='./logs/', row=2, weight_path='./weights/SHTechA.pth')
File "run_test.py", line 140, in
main(args)
File "run_test.py", line 107, in main
result = self.forward(*input, **kwargs)
File "D:\working_space_HJ\CrowdCounting-P2PNet-main\models\p2pnet.py", line 215, in forward
features_fpn = self.fpn([features[1], features[2], features[3]])
File "C:\ProgramData\Anaconda3\envs\crowd\lib\site-packages\torch\nn\modules\module.py", line 550, in call
result = self.forward(*input, **kwargs)
File "D:\working_space_HJ\CrowdCounting-P2PNet-main\models\p2pnet.py", line 183, in forward
P4_x = P5_upsampled_x + P4_x
RuntimeError: The size of tensor a (134) must match the size of tensor b (135) at non-singleton dimension 2

please help me and I'm really appreciate it if anyone could answer these questions. Thanks very much. ^ ^

scene01，scene02这些怎么处理。比如shanghaiA
train.txt test.txt

您好，非常感谢您的代码，我在使用的时候有个问题，使用如下代码：
“CUDA_VISIBLE_DEVICES=0 python run_test.py --weight_path ./weights/SHTechA.pth --output_dir ./logs/”执行后得到：“Namespace(backbone='vgg16_bn', gpu_id=0, line=2, output_dir='./logs/', row=2, weight_path='./weights/SHTechA.pth')”
后面就没有反应了，代码没有任何更改。

This is very meaningful work.

Can anyone reproduce the accuracy of UCF-QNRF? I hava conducted acout ten experiments on UCF-QNRF, however, the MAE and RMSE is 100 and 180. I can not achieve the reported MAE:85 and RMSE:154.5, even I use the settings in this paper.

RuntimeError: CUDA out of memory. Tried to allocate 88.00 MiB (GPU 0; 8.00 GiB total capacity; 6.17 GiB already allocated; 63.69 MiB free; 6.23 GiB reserved in total by PyTorch)

Hi,

I could reproduce MAE results on ShanghaiTech part A using supplied arguments for train script. However, with same parameters, for ShanghaiTech part B I can not reproduce results, getting a MAE of 11.18 instead of article results which get a 6.25 .

I have created an script to build train files out of ShanghaiTech datasets and I can confirm that this part is ok because I can reproduce MAE with generated files (for part A).

For partB, I have tried with same supplied parameters and also changing learning rate but MAE does not go below 11.

Any more tweaks to be done in dataset loading? Any different parameter values?

Many thanks for your great article and code,

Ricard

While reading the paper, I came to know that the authors used 13 convs from VGG16 network to extract deep features. Since VGG nets have been around for a quite a long time, why didn't you choose some more efficient and accurate networks? Or is there something I don't know of? Thank you.

Hi!

How do one add support training on images without heads? Is it possible?

I found
loss_dict_reduced_scaled = {k: v * weight_dict[k] for k, v in loss_dict_reduced.items() if k in weight_dict}
the key in weight_dict and loss_dict_reduced is not consistent with each other.
one is loss_point, the other is ;oss_points.

In models/p2pnet.py:

Line 278: losses['loss_point'] = loss_bbox.sum() / num_points
Line 335: weight_dict = {'loss_ce': 1, 'loss_points': args.point_loss_coef}

one is "loss_points" and another is "loss_point" (an "s" is different) ! This would cause that the point loss is actually not used, which is fully wrong!

Awesome work!
But it seems that the distributed training doesn't work, i.e., the argument --gpu_id is unused.

这里是typo？应该是loss_points吧

Is it possible to use it for other tasks such as animal detection or objects other than human?

Please let us know your parameter settings of QNRF to achieve the results reported in the paper.
Thank you in advance.

请问将训练结果如何转换onnx格式

你好，请问我使用ShanghaiTech_Crowd_Counting_Dataset/part_A_final数据进行训练，参数使用的是readme里面的参数，但是精度很差，是什么原因？
2779][lr 0.0001000][11.72s]mae:329.14285714285717, mse:466.4266075383996, time:10.338678121566772, best mae:295.75274725274727metric/mae@2778: 329.14285714285717metric/mse@2778: 466.4266075383996loss/loss@2780: 0.04982551398712236loss/loss_ce@2780: 0.04982551398712236[ep 2780][lr 0.0001000][12.00s]

I am getting below error while running this on CPU machine(i haven't installed Cuda enabled Pytorch as my computer don't have GPU)

AssertionError: Torch not compiled with CUDA enabled

Very nice work.
I want to know wehre can obtain the evaluation code of nAP.

Thank you for your great code and algorisms.
I'm trying to use this code, but still not work.

When I run train.py, I got this massage as blow.

Namespace(backbone='vgg16_bn', batch_size=8, checkpoints_dir='./ckpt', clip_max_norm=0.1, data_root='/content/drive/MyDrive/DATA_ROOT', dataset_file='SHHA', eos_coef=0.5, epochs=3500, eval=False, eval_freq=5, frozen_weights=None, gpu_id=0, line=2, lr=0.0001, lr_backbone=1e-05, lr_drop=3500, num_workers=8, output_dir='./logs', point_loss_coef=0.0002, resume='', row=2, seed=42, set_cost_class=1, set_cost_point=0.05, start_epoch=0, tensorboard_dir='./runs', weight_decay=0.0001)
number of params: 21579344
Start training
Averaged stats: Traceback (most recent call last):
File "train.py", line 222, in
main(args)
File "train.py", line 161, in main
args.clip_max_norm)
File "/content/CrowdCounting-P2PNet/engine.py", line 120, in train_one_epoch
print("Averaged stats:", metric_logger)
File "/content/CrowdCounting-P2PNet/util/misc.py", line 186, in str
"{}: {}".format(name, str(meter))
File "/content/CrowdCounting-P2PNet/util/misc.py", line 85, in str
median=self.median,
File "/content/CrowdCounting-P2PNet/util/misc.py", line 64, in median
return d.median().item()
RuntimeError: median cannot be called with empty tensor

I seem it cannot read dataset from list file.
my train.list is
train/RIMG6463.JPG train/RIMG6463.txt
train/RIMG6464.JPG train/RIMG6464.txt
train/RIMG6465.JPG train/RIMG6465.txt
train/RIMG6466.JPG train/RIMG6466.txt
train/RIMG6467.JPG train/RIMG6467.txt

that under DATA_ROOT.
What is wrong about this?

Thank you for your help.

Hi all!

I'd love to use this model in our ONNX flows but wasn't able to convert it to ONNX. Is there any known way of converting this model to ONNX?

Code I am using:

import os
import sys
import torch

sys.path.append(os.path.abspath(f"{os.getcwd()}/model"))

# Available after the above append
# it's in the model folder
from model.models.p2pnet import P2PNet
from model.models.backbone import Backbone_VGG

def main():
    onnx_model_name = sys.argv[1] or "model"
    onnx_model_name = f"{onnx_model_name}.onnx"

    print("Loading Model")
    # Create the model
    model_backbone = Backbone_VGG("vgg16_bn", True)
    model = P2PNet(model_backbone, 2, 2)

    # Load Weights
    checkpoint = torch.load("./model/weights/SHTechA.pth", map_location=torch.device('cpu'))
    model.load_state_dict(checkpoint["model"])
    model.eval() # Put in inference mode
    
    # Create dummy input
    dummy_input = torch.randn(1, 3, 640, 640)
    # dummy_input1 = torch.randn(1, 3, 1024, 1024)
    # dummy_input = (dummy_input0, dummy_input1)

    # Export as ONNX
    print(f"Exporting as ONNX: {onnx_model_name}")
    torch.onnx._export(
        model,
        dummy_input,
        onnx_model_name, # Output name
        opset_version=13, # ONNX Opset Version
        export_params=True, # Store the trained parameters in the model file
        do_constant_folding=True, # Execute constant folding for optimization
        input_names = ['input'],   # the model's input names 
        # output_names = ['pred_logits', 'pred_points'], # the model's output names (see forward in the architecture)
        output_names = ['pred_logits', 'pred_points'], # the model's output names (see forward in the architecture)
        dynamic_axes={
            # Input is an image [batch_size, channels, width, height]
            # all of it can be variable so we need to add it in dynamic_axes
            'input': {
                0: 'batch_size',
                1: 'channels',
                2: 'width',
                3: 'height'
            }, 
            'pred_logits': [0, 1, 2],
            'pred_points': [0, 1, 2],
        } 
    )


if __name__ == "__main__":
    main()

Error I receive:

[ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running Add node. Name:'Add_88' Status Message: Add_88: right operand cannot broadcast on dim 1 LeftShape: {1,40960,2}, RightShape: {1,25600,2}

训练过程中，显存会随着epoch忽高忽低。有时会导致显卡爆炸。
有人遇到这种问题吗？

    您好，我也是一样的情况，在出现了Namespace(backbone='vgg16_bn', gpu_id=0, line=2, output_dir='./logs/', row=2, weight_path='./weights/SHTechA.pth')之后，后面就没有反应了。不知道有没有顺利解决这个问题的

Originally posted by @fatakWang in #16 (comment)

我在试着复现程式
请问regression head的部分是如何建构的?
以及regression loss在没有权重调整前数值大概是多少?

Hello, thank you for this project. My question is, how can I have a video input instead of a single image? I would like to test it with a video or a webcam in real time.

Why I tested the best SHTechA.pth and got a result of 53 MAE?

Hello,

thanks for uploading the code!

When trying to run a test using supplied pretrained Shanghai Dataset Part A, an error is raised about VGG_BN pretrained weights:

No such file or directory: '/apdcephfs/private_changanwang/checkpoints/vgg16_bn-6c64b313.pth

Could you share it?

Thanks

论文的结果非常impressive，不知道作者什么时候可以release代码让大家共同学习一下呢？

如题

我直接训练最好得到54，但是我用给出的权重（51.9，阈值保持默认为0.5）跑出来是53.13，但是阈值改成0.45是可以到52的，请问51.9是在当前代码默认setting下训练出来的还是有所调整呢？
如果是在当前代码下跑出来的，请问能给一下你们的SHTechA的数据集和组织格式吗？因为我只改了loader。

你好，何时开放training代码，每天都在盼星星盼月亮地期待着。

Originally posted by @Tracyummy in #7 (comment)

有没有朋友在这几个数据集上实验过，什么样的参数设置可以取得比较好的结果？感谢分享

Hi everyone, 你們好,

When I run the train.py, it occurs the following problems:
當我運行train.py時,出現了以下問題:

The annotation points in the Txt file follow as instructions.
每一張圖片的annotation points 都是根據要求寫的

x1 y1
x2 y2

Each image will contain at least 4 points
每張圖片至少有4個點

Some of the annotation points and images have been done.
已經有一部份圖片處理了.

Just wondering how to solve this problem.
請問一下我要如何解決這個問題?

感謝感謝

can anyone please guide me where to get ShanghaiTechA dataset in a format given in repository, as I find the dataset containing .mat file.