OSError: [Errno 12] Cannot allocate memory about upsnet

[YuwenXiong]

I think that might because your cpu memory is not large enough. You can modify base_dataset.py L225, change cpu_num to a smaller number, e.g. 1, 2 or 4

[WenFuLee]

I tried cpu_num = 1 and 2, but it didn't help. Wondering how many CPU ram you have.
I google this issue: https://stackoverflow.com/questions/20111242/how-to-avoid-errno-12-cannot-allocate-memory-errors-caused-by-using-subprocess/26416661
I may try to use swap now...

[YuwenXiong]

Then please try to remove all multi-thread code, use a single core instead, for example change L228 to p = BaseDataset._load_image_single_core(proc_id, files_set, pan_gt_folder) and do corresponding modification for other part.

[WenFuLee]

After adding swap, I got some progress. I haven't tried to remove all multi-thread code and to use a single core. All I changed is to hard-code cpu_num to 1. Do you have any idea? Thanks.

2019-04-17 04:52:53,975 | upsnet_end2end_test.py | line 306: unified pano result:
Traceback (most recent call last):
  File "upsnet/upsnet_end2end_test.py", line 315, in <module>
    upsnet_test()
  File "upsnet/upsnet_end2end_test.py", line 307, in upsnet_test
    test_dataset.evaluate_panoptic(test_dataset.get_unified_pan_result(all_ssegs, all_panos, all_pano_cls_inds, stuff_area_limit=config.test.panoptic_stuff_area_limit), os.path.join(final_output_path, 'results', 'pans_unified'))
  File "upsnet/../upsnet/dataset/base_dataset.py", line 327, in evaluate_panoptic
    pred_pans, pred_json = get_pred(pred_pans_2ch, color_gererator)
  File "upsnet/../upsnet/dataset/base_dataset.py", line 257, in get_pred
    p = p.get()
  File "/home/wen-fulee/anaconda3/lib/python3.6/multiprocessing/pool.py", line 644, in get
    raise self._value
  File "/home/wen-fulee/anaconda3/lib/python3.6/multiprocessing/pool.py", line 424, in _handle_tasks
    put(task)
  File "/home/wen-fulee/anaconda3/lib/python3.6/multiprocessing/connection.py", line 206, in send
    self._send_bytes(_ForkingPickler.dumps(obj))
  File "/home/wen-fulee/anaconda3/lib/python3.6/multiprocessing/connection.py", line 393, in _send_bytes
    header = struct.pack("!i", n)
struct.error: 'i' format requires -2147483648 <= number <= 2147483647

[WenFuLee]

After removing all multi-thread code and using a single core instead, I got errors about p.get(),
which shows p is a list object and doesn't have get() function. How should I modify codes for p.get() as well? Thanks.

[YuwenXiong]

use p instead

[WenFuLee]

This issue was solved after removing all multi-thread code and using a single core instead.

[phongnhhn92]

@WenFuLee hi ! can you share the base_dataset.py that you have modified to run test on a single GPU ?

[ChengJianjia]

This issue was solved after removing all multi-thread code and using a single core instead.
can you share how to remove all multi-thread code?