3+ confd cluster configuration does not work about cortx-motr

[stale]

This issue/pull request has been marked as needs attention as it has been left pending without new activity for 4 days. Tagging @huanghua78 @mukundkanekar for appropriate assignment. Sorry for the delay & Thank you for contributing to CORTX. We will get back to you as soon as possible.

[andriytk]

I could not reproduce the issue on my local 7-nodes cluster (+1 cmu +1 client node). I configured 3 confd services and could run i/o with such configuration:

[vagrant@cmu ~]$ hctl status
Data pool:
    # fid name
    0x6f00000000000001:0x11b 'default'
Profile:
    # fid name: pool(s)
    0x7000000000000001:0x199 'all': 'default' None None
Services:
    client1.local
    [started]  hax        0x7200000000000001:0x10f  192.168.85.141@tcp:12345:1:1
    [unknown]  m0_client  0x7200000000000001:0x112  192.168.85.141@tcp:12345:4:1
    [unknown]  m0_client  0x7200000000000001:0x115  192.168.85.141@tcp:12345:4:2
    [unknown]  m0_client  0x7200000000000001:0x118  192.168.85.141@tcp:12345:4:3
    ssu1.local
    [started]  hax        0x7200000000000001:0x16  192.168.85.133@tcp:12345:1:1
    [started]  confd      0x7200000000000001:0x19  192.168.85.133@tcp:12345:2:1
    [started]  ioservice  0x7200000000000001:0x1c  192.168.85.133@tcp:12345:2:2
    [unknown]  m0_client  0x7200000000000001:0x30  192.168.85.133@tcp:12345:4:1
    [unknown]  m0_client  0x7200000000000001:0x33  192.168.85.133@tcp:12345:4:2
    [unknown]  m0_client  0x7200000000000001:0x36  192.168.85.133@tcp:12345:4:3
    cmu.local  (RC)
    [started]  hax        0x7200000000000001:0x4   192.168.85.132@tcp:12345:1:1
    [started]  confd      0x7200000000000001:0x7   192.168.85.132@tcp:12345:2:1
    [unknown]  m0_client  0x7200000000000001:0xa   192.168.85.132@tcp:12345:4:1
    [unknown]  m0_client  0x7200000000000001:0xd   192.168.85.132@tcp:12345:4:2
    [unknown]  m0_client  0x7200000000000001:0x10  192.168.85.132@tcp:12345:4:3
    ssu2.local
    [started]  hax        0x7200000000000001:0x3c  192.168.85.134@tcp:12345:1:1
    [started]  confd      0x7200000000000001:0x3f  192.168.85.134@tcp:12345:2:1
    [started]  ioservice  0x7200000000000001:0x42  192.168.85.134@tcp:12345:2:2
    [unknown]  m0_client  0x7200000000000001:0x56  192.168.85.134@tcp:12345:4:1
    [unknown]  m0_client  0x7200000000000001:0x59  192.168.85.134@tcp:12345:4:2
    [unknown]  m0_client  0x7200000000000001:0x5c  192.168.85.134@tcp:12345:4:3
    ssu3.local
    [started]  hax        0x7200000000000001:0x62  192.168.85.135@tcp:12345:1:1
    [started]  ioservice  0x7200000000000001:0x65  192.168.85.135@tcp:12345:2:1
    [unknown]  m0_client  0x7200000000000001:0x79  192.168.85.135@tcp:12345:4:1
    [unknown]  m0_client  0x7200000000000001:0x7c  192.168.85.135@tcp:12345:4:2
    [unknown]  m0_client  0x7200000000000001:0x7f  192.168.85.135@tcp:12345:4:3
    ssu4.local
    [started]  hax        0x7200000000000001:0x85  192.168.85.136@tcp:12345:1:1
    [started]  ioservice  0x7200000000000001:0x88  192.168.85.136@tcp:12345:2:1
    [unknown]  m0_client  0x7200000000000001:0x9c  192.168.85.136@tcp:12345:4:1
    [unknown]  m0_client  0x7200000000000001:0x9f  192.168.85.136@tcp:12345:4:2
    [unknown]  m0_client  0x7200000000000001:0xa2  192.168.85.136@tcp:12345:4:3
    ssu5.local
    [started]  hax        0x7200000000000001:0xa8  192.168.85.137@tcp:12345:1:1
    [started]  ioservice  0x7200000000000001:0xab  192.168.85.137@tcp:12345:2:1
    [unknown]  m0_client  0x7200000000000001:0xbf  192.168.85.137@tcp:12345:4:1
    [unknown]  m0_client  0x7200000000000001:0xc2  192.168.85.137@tcp:12345:4:2
    [unknown]  m0_client  0x7200000000000001:0xc5  192.168.85.137@tcp:12345:4:3
    ssu6.local
    [started]  hax        0x7200000000000001:0xcb  192.168.85.138@tcp:12345:1:1
    [started]  ioservice  0x7200000000000001:0xce  192.168.85.138@tcp:12345:2:1
    [unknown]  m0_client  0x7200000000000001:0xe2  192.168.85.138@tcp:12345:4:1
    [unknown]  m0_client  0x7200000000000001:0xe5  192.168.85.138@tcp:12345:4:2
    [unknown]  m0_client  0x7200000000000001:0xe8  192.168.85.138@tcp:12345:4:3
    ssu7.local
    [started]  hax        0x7200000000000001:0xee  192.168.85.139@tcp:12345:1:1
    [started]  ioservice  0x7200000000000001:0xf1  192.168.85.139@tcp:12345:2:1
    [unknown]  m0_client  0x7200000000000001:0x105  192.168.85.139@tcp:12345:4:1
    [unknown]  m0_client  0x7200000000000001:0x108  192.168.85.139@tcp:12345:4:2
    [unknown]  m0_client  0x7200000000000001:0x10b  192.168.85.139@tcp:12345:4:3
[vagrant@cmu ~]$

i/o:

[vagrant@cmu mcp]$ ./mcp -prof 0x7000000000000001:0x199 -hax 192.168.85.132@tcp:12345:1:1 -proc 0x7200000000000001:0xa -ep 192.168.85.132@tcp:12345:4:1 -osz $((128*1024)) -v -bsz 128 -threads 4 /dev/zero 12345:123456703
2020/12/14 16:32:38 mio.go:477: W: off=0 len=134217728 bs=134217728 gs=33554432 speed=8364 (Kbytes/sec)
2020/12/14 16:32:54 mio.go:477: W: off=134217728 len=134217728 bs=134217728 gs=33554432 speed=8327 (Kbytes/sec)
2020/12/14 16:33:09 mio.go:477: W: off=268435456 len=134217728 bs=134217728 gs=33554432 speed=8396 (Kbytes/sec)
2020/12/14 16:33:25 mio.go:477: W: off=402653184 len=134217728 bs=134217728 gs=33554432 speed=8255 (Kbytes/sec)
^C

CDF - cmu-ssu7-cleint1-cdf.yaml.txt

Versions:

• Motr - commit fa45ae7;
• Hare - commit 571ae39.

Will try it again on the SAGE prototype...

[andriytk]

Will close it for now.