Discrepancies in Reproducing Results for TSP Diffusion Method about difusco

[Edward-Sun]

Hi @xyfffff ,

Thanks a lot for reporting the issue. I checked the log and just realized the tsp-10000-mcts results are obtained by applying a 2-OPT process before feeding to the MCTS solver. Please check the updated code:

https://github.com/Edward-Sun/DIFUSCO/blob/main/tsp_mcts/solve-10000.sh

https://github.com/Edward-Sun/DIFUSCO/blob/main/tsp_mcts/convert_numpy_with_2opt.py

[Edward-Sun]

Hi @xyfffff ,

Thank you for closely following our work. I found my notes wrote some like

# Mean (PDM + 0.01 * dist scores): 73.8229338125
# Mean (PDM + 2OPT + 0.01 * dist scores): 73.621185875

So I'm sure the final results are reported with 2-opt. I will check the code again soon.

[xyfffff]

It's quite strange that now I rerun the script and got the result of SL+MCTS for TSP-500 to be 16.62 (which is better than the result reported in the paper...), and the result of SL+MCTS for TSP-10000 to be 73.76 (which is better than my previous result but still worse than the result in the paper...).

[xyfffff]

Dear Edward,

I hope this message finds you well. First and foremost, I would like to express my gratitude for your contributions to the field and for sharing your code on GitHub. It's a valuable resource for the community.

I've recently tried running your updated script and would like to share some observations and seek your assistance for a few issues I encountered:

1. Performance Concerns:

• For TSP-10000, the script seems quite time-consuming during the merge process and the 2-opt application. It takes approximately 33 seconds for Union find and 205 seconds for 2-opt per instance. This duration exceeds the time reported in the original paper's results, leading to longer overall computation times for the model.

2. Issue with 2-opt Implementation:

• After applying 2-opt, I observed a notable decrease in performance. Specifically, the average MCTS length increased to 79.39, which is significantly worse than the result before the 2-opt modification.

Additionally, I encountered some minor issues while trying to run the updated code directly:

• Imports Adjustment:

import sys
sys.path.append('../difusco')

from utils.tsp_utils import TSPEvaluator
from utils.cython_merge.cython_merge import merge_cython

• Initialization of solved_costs:
  It appears that solved_costs = [] needs to be defined in the main function.
• Misplaced TSPEvaluator:
  The TSPEvaluator is defined in tsp_utils.py rather than TSP_2opt.
• Formatting in Main Function:
  There seem to be some inaccuracies with the indentations in the main function.

I highly appreciate your efforts and would be immensely grateful for any guidance or suggestions you could provide to help me accurately replicate your results.

Thank you for your time and assistance.

Best regards,
Yifan