ibaigorordo / depthai-topformer-semantic-segmentation

Python scripts performing on devive semantic segmentation using the TopFormer model in depthai.

depthai-topformer-semantic-segmentation's Issues

Could you share your original onnx?

Many thanks to your excellent work

Have you try to convert segformer pytorch in transformers library to depthai, I am failed to convert the segformer to depthai.

Issues running the Colab notebook

I was trying to run the Google Colab notebook in order to compile a MyriadX Blob with 3 shaves, but ran into numerous package compatibility problems — some of which are down to the installed version of PyTorch, which I wasn't able to correct. After a lot of trial-and-error I was able to perform the compilation procedure locally, see the script below for reference:

# For some reason this line fails in Colab, which is why I decided to run the compilation procedure locally.
pip install --extra-index-url https://download.pytorch.org/whl/lts/1.8/cu102 \
torch==1.8.2 \
torchvision==0.9.2 \
torchaudio==0.8.2 

pip install -U mmcv-full==1.3.13 mmsegmentation==0.18.0

pip install gdown

model_names=( \
 "TopFormer-T_448x448_2x8_160k" \
 "TopFormer-T_448x448_4x8_160k" \
 "TopFormer-T_512x512_2x8_160k" \
 "TopFormer-T_512x512_4x8_160k" \
 "TopFormer-S_512x512_2x8_160k" \
 "TopFormer-S_512x512_4x8_160k" \
 "TopFormer-B_512x512_2x8_160k" \
 "TopFormer-B_512x512_4x8_160k" \
)

config_names=( \
 "topformer_tiny_448x448_160k_2x8_ade20k.py" \
 "topformer_tiny_448x448_160k_2x8_ade20k.py" \
 "topformer_tiny_512x512_160k_2x8_ade20k.py" \
 "topformer_tiny_512x512_160k_2x8_ade20k.py" \
 "topformer_small_512x512_160k_2x8_ade20k.py" \
 "topformer_small_512x512_160k_2x8_ade20k.py" \
 "topformer_base_512x512_160k_2x8_ade20k.py" \
 "topformer_base_512x512_160k_4x8_ade20k.py"
)

model_gdrive_ids=( \
 "1HsdinQHVKGzi8INJy29L48Ho1MmIeCZN" \
 "1UW6kmt5oxssxHylGaZg5CByP7xcFrOcF" \
 "11mxM2roRD-wXRYjw7YzKQVbQs2jsvYYw" \
 "1OnS3_PwjJuNMWCKisreNxw_Lma8uR8bV" \
 "16TJLt_lxJ-QvrDmYvhY_6vxRCYpyIN1n" \
 "19041fMb4HuDyNhIYdW1r5612FyzpexP0" \
 "1pYUUB4N6FVjXt1NKygtQis5WmhGECrNt" \
 "1m7CxYKWAyJzl5W3cj1vwsW4DfqAb_rqz" \
)

# Input image dimensions, (un)comment as needed.
input_width='448'
input_height='448'

# Input image dimensions, (un)comment as needed.
#input_width='512'
#input_height='512'

# Change the index to select the desired model.
model_index=0 

model_name=${model_names[${model_index}]}
config_name=${config_names[${model_index}]}
config_path="local_configs/topformer/${config_name}"
model_gdrive_id=${model_gdrive_ids[${model_index}]}

gdown "https://drive.google.com/uc?id=${model_gdrive_id}" -O "${model_name}".pth

python3 "tools/convert2onnx.py" \
"${config_path}" \
--input-img demo/demo.png \
--shape "${input_height}" "${input_width}" \
--checkpoint "${model_name}".pth \
--output-file "${model_name}".onnx

wget https://github.com/PINTO0309/tflite2tensorflow/releases/download/v1.20.8/l_openvino_toolkit_p_2021.4.582.tgz
tar xf l_openvino_toolkit_p_2021.4.582.tgz

cd l_openvino_toolkit_p_2021.4.582

sudo -E ./install_openvino_dependencies.sh
sed -i 's/decline/accept/g' silent.cfg
sudo ./install.sh --silent silent.cfg

cd ..

source /opt/intel/openvino_2021/bin/setupvars.sh
python /opt/intel/openvino_2021/deployment_tools/model_optimizer/mo.py \
--input_model  "${model_name}".onnx \
--model_name topformer \
--data_type FP16 \
--output_dir openvino/ \
--input_shape [1,3,"${input_height}","${input_width}"] \
--mean_values="[123.675, 116.28 , 103.53]" \
--scale_values="[58.395, 57.12 , 57.375]"

blobconverter \
--openvino-bin "openvino/topformer.bin" \
--openvino-xml "openvino/topformer.xml" \
--data-type "FP16" \
--shaves 3 \
--version '2021.4'

Could you tell us how to convert the blob files

Hi, there, I am not familiar with mmseg, could you help me fix a issue.
I trained the topformer model with my customer dataset and the output is one hot code embedding (150X512X512), how could we output the predict labels seg map (512X512)?