HPX error: "Host not found" when running on Expanse with 128 nodes about hpx

[hkaiser]

Does this also happen with the TCP parcel port being enabled? Also, could you provide a stack backtrace to the point where the exception is thrown?

[JiakunYan]

Yes, it also happens with TCP parcelport enabled. HPX did not give me a stack backtrace after the exception. It seems the program just hanged and was then killed by slurm.

[JiakunYan]

Tried running the program with two nodes using --hpx:debug-clp. Here is a snippet of the logging I got:

SLURM nodelist found (SLURM_STEP_NODELIST): exp-4-[08-09]
SLURM nodelist found (SLURM_STEP_NODELIST): exp-4-[08-09]
batch_name: SLURM
batch_name: SLURM
num_threads: 128
node_num_: 1
num_threads: 128
num_localities: 2
got node list
node_num_: 0
num_localities: 2
extracted: 'exp-4-08'
got node list
extracted: 'exp-4-08'
incrementing agas_node_num
extracted: 'exp-4-09'
incrementing agas_node_num
extracted: 'exp-4-09'
incrementing agas_node_num
incrementing agas_node_num
using AGAS host: 'exp-4-08' (node number 0)
Nodes from nodelist:
using AGAS host: 'exp-4-08' (node number 0)
Nodes from nodelist:
exp-4-08: 1 (10.21.4.8:0)
exp-4-09: 1 (10.21.4.9:0)
exp-4-08: 1 (198.202.103.67:0)
agas host_name: exp-4-08
exp-4-09: 1 (198.202.103.66:0)
agas host_name: exp-4-08
asio host_name: exp-4-09
asio host_name: exp-4-08
host_name: exp-4-09
host_name: exp-4-08
resolved: 'exp-4-09' to: 198.202.103.66
resolved: 'exp-4-08' to: 198.202.103.67
resolved: 'exp-4-08' to: 198.202.103.67