Run stratum failed on brcm Tomahawk platfrom about stratum

[pudelkoM]

I see that the crash happens in Hal::Run(), but that's about it. Can you run a stratum build with debug information?
bazel build --config=asan //stratum/hal/bin/bcm/standalone:stratum_bcm

If this does not build, try adding these flags to .bazelrc under general build options:

build --copt -O1
build --copt -g
build --copt -fno-omit-frame-pointer

[bbinxie]

Thank you for help.
Just have confused, i used sdklt lib is build from master branch with kernel 3.16, So I want know is it right?
Or Should i used sdklt lib from r78 release version?

I have build success follow your step. As below running log,

Message from syslogd@localhost at Dec 5 17:10:54 ...
stratum_bcm[3392]: INFO main.cc:89] Starting Stratum in STANDALONE mode for a Broadcom-based switch...
I1205 17:10:54.842617 3392 timer_daemon.cc:101] The timer daemon has been started.

Message from syslogd@localhost at Dec 5 17:10:54 ...
stratum_bcm[3392]: INFO timer_daemon.cc:101] The timer daemon has been started.
I1205 17:10:54.843001 3392 hal.cc:133] HAL sanity checks all passed.
I1205 17:10:54.843129 3392 hal.cc:139] Setting up HAL in COLDBOOT mode...

Message from syslogd@localhost at Dec 5 17:10:54 ...
stratum_bcm[3392]: INFO hal.cc:133] HAL sanity checks all passed.

Message from syslogd@localhost at Dec 5 17:10:54 ...
stratum_bcm[3392]: INFO hal.cc:139] Setting up HAL in COLDBOOT mode...
I1205 17:10:54.843580 3392 config_monitoring_service.cc:116] Pushing the saved chassis config read from /root/sdklt/config/slx/slx_chassis_config.pb.txt...

Message from syslogd@localhost at Dec 5 17:10:54 ...
stratum_bcm[3392]: INFO config_monitoring_service.cc:116] Pushing the saved chassis config read from /root/sdklt/config/slx/slx_chassis_config.pb.txt...
I1205 17:10:54.866163 3392 bcm_switch.cc:492] Chassis config verified successfully.

Message from syslogd@localhost at Dec 5 17:10:54 ...
stratum_bcm[3392]: INFO bcm_switch.cc:492] Chassis config verified successfully.
I1205 17:10:54.867209 3392 bcm_chassis_manager.cc:1020] Portmap:
Panel, logical (PORT_ID), physical (PC_PHYS_PORT_ID)
1, 1, 65
2, 2, 69

Message from syslogd@localhost at Dec 5 17:10:54 ...
stratum_bcm[3392]: INFO bcm_chassis_manager.cc:1020] Portmap:#012Panel, logical (PORT_ID), physical (PC_PHYS_PORT_ID)#12 1, 1, 65#012 2, 2, 69
I1205 17:10:54.867748 3392 bcm_sdk_wrapper.cc:817] Found 1 device.

Message from syslogd@localhost at Dec 5 17:10:54 ...
stratum_bcm[3392]: INFO bcm_sdk_wrapper.cc:817] Found 1 device.
I1205 17:10:55.872360 3392 bcm_sdk_wrapper.cc:884] Unit 0 is assigned to SOC device BCM56960 found on PCI bus 1, PCI slot 0.

Message from syslogd@localhost at Dec 5 17:10:55 ...
stratum_bcm[3392]: INFO bcm_sdk_wrapper.cc:884] Unit 0 is assigned to SOC device BCM56960 found on PCI bus 1, PCI slot 0.
I1205 17:10:56.160243 3392 bcm_sdk_wrapper.cc:1338] Unit 0 initialized successfully (warm_boot: NO).

Message from syslogd@localhost at Dec 5 17:10:56 ...
stratum_bcm[3392]: INFO bcm_sdk_wrapper.cc:1338] Unit 0 initialized successfully (warm_boot: NO).
BCMLT.0> I1205 17:10:56.935297 3392 bcm_chassis_manager.cc:2040] Successfully set the following options for SingletonPort (slot: 1, port: 1, unit: 0, logical_port: 1, speed: 100G): (enabled: false, blocked: true)

Message from syslogd@localhost at Dec 5 17:10:56 ...
stratum_bcm[3392]: INFO bcm_chassis_manager.cc:2040] Successfully set the following options for SingletonPort (slot: 1, port: 1, unit: 0, logical_port: 1, speed: 100G): (enabled: false, blocked: true)
I1205 17:10:57.654875 3392 bcm_chassis_manager.cc:2040] Successfully set the following options for SingletonPort (slot: 1, port: 2, unit: 0, logical_port: 2, speed: 100G): (enabled: false, blocked: true)

Message from syslogd@localhost at Dec 5 17:10:57 ...
stratum_bcm[3392]: INFO bcm_chassis_manager.cc:2040] Successfully set the following options for SingletonPort (slot: device knet-0-0 entered promiscuous mode
1, port: 2, unit: 0, logical_port: 2, speed: 100G): (enabled: false, blocked: true)
I1205 17:10:57.656108 3392 bcm_sdk_wrapper.cc:4120] Added VLAN 2 on unit 0.

Message from syslogd@localhost at Dec 5 17:10:57 ...
stratum_bcm[3392]: INFO bcm_sdk_wrapper.cc:4120] Added VLAN 2 on unit 0.
I1205 17:10:57.656261 3396 bcm_sdk_wrapper.cc:236] Unit: 0 Port: 1 Link: DOWN.
I1205 17:10:57.656728 3396 bcm_sdk_wrapper.cc:236] Unit: 0 Port: 2 Link: DOWN.
I1205 17:10:57.656615 3392 bcm_sdk_wrapper.cc:4204] VLAN profile 2 does not exist.

Message from syslogd@localhost at Dec 5 17:10:57 ...
stratum_bcm[3392]: INFO bcm_sdk_wrapper.cc:236] Unit: 0 Port: 1 Link: DOWN.

Message from syslogd@localhost at Dec 5 17:10:57 ...
stratum_bcm[3392]: INFO bcm_sdk_wrapper.cc:236] Unit: 0 Port: 2 Link: DOWN.

Message from syslogd@localhost at Dec 5 17:10:57 ...
stratum_bcm[3392]: INFO bcm_sdk_wrapper.cc:4204] VLAN profile 2 does not exist.
I1205 17:10:57.657279 3392 bcm_sdk_wrapper.cc:4258] VLAN 2 has VLAN profile 2

Message from syslogd@localhost at Dec 5 17:10:57 ...
stratum_bcm[3392]: INFO bcm_sdk_wrapper.cc:4258] VLAN 2 has VLAN profile 2
I1205 17:10:57.657722 3392 bcm_sdk_wrapper.cc:2370] Created a new L3 drop egress intf: (port: 0, vlan: 1, router_intf_id: 0, dst_mac: 00:00:00:00:00:00, egress_intf_id: 1) on unit 0.

Message from syslogd@localhost at Dec 5 17:10:57 ...
stratum_bcm[3392]: INFO bcm_sdk_wrapper.cc:2370] Created a new L3 drop egress intf: (port: 0, vlan: 1, router_intf_id: 0, dst_mac: 00:00:00:00:00:00, egress_intf_id: 1) on unit 0.
W1205 17:10:57.658664 3392 bcm_sdk_wrapper.cc:4824] Currently not explicitly enabling/disabling ACL stages for packets ingressing on internal and external ports.
W1205 17:10:57.659157 3392 bcm_sdk_wrapper.cc:4839] Enabling intra-slice double wide is not supported.
W1205 17:10:57.659269 3392 bcm_sdk_wrapper.cc:4844] Stats collection hardware read-through is not supported.
I1205 17:10:57.659405 3392 bcm_acl_manager.cc:375] ACL manager successfully configured ACLs for node with ID 1 mapped to unit 0.

Message from syslogd@localhost at Dec 5 17:10:57 ...
stratum_bcm[3392]: INFO bcm_acl_manager.cc:375] ACL manager successfully configured ACLs for node with ID 1 mapped to unit 0.
I1205 17:10:57.743196 3392 bcm_packetio_manager.cc:758] KNET interface knet-0-0 created for node with ID 1 (unit: 0, purpose: BCM_KNET_INTF_PURPOSE_CONTROLLER, vlan: 1, cpu_queue: 0, netif_id: 1, netif_index: 38, rx_thread_id: 140112485033728).
I1205 17:10:57.743386 3392 bcm_switch.cc:76] Chassis config pushed successfully.

Message from syslogd@localhost at Dec 5 17:10:57 ...
stratum_bcm[3392]: INFO bcm_packetio_manager.cc:758] KNET interface knet-0-0 created for node with ID 1 (unit: 0, purpose: BCM_KNET_INTF_PURPOSE_CONTROLLER, vlan: 1, cpu_queue: 0, netif_id: 1, netif_index: 38, rx_thread_id: 140112485033728).
I1205 17:10:57.743527 3392 gnmi_events.h:618] Handling N7stratum3hal24ConfigHasBeenPushedEventE
I1205 17:10:57.743649 3392 gnmi_publisher.h:220] Configuration has changed.
I1205 17:10:57.743710 3413 bcm_chassis_manager.cc:1755] State of SingletonPort (node_id: 1, port_id: 1, slot: 1, port: 1, unit: 0, logical_port: 1, speed: 100G): DOWN

Message from syslogd@localhost at Dec 5 17:10:57 ...
stratum_bcm[3392]: INFO bcm_switch.cc:76] Chassis config pushed successfully.

Message from syslogd@localhost at Dec 5 17:10:57 ...
stratum_bcm[3392]: INFO gnmi_events.h:618] Handling N7stratum3hal24ConfigHasBeenPushedEventE

Message from syslogd@localhost at Dec 5 17:10:57 ...
stratum_bcm[3392]: INFO gnmi_publisher.h:220] Configuration has changed.
I1205 17:10:57.744055 3413 bcm_chassis_manager.cc:1755] State of SingletonPort (node_id: 1, port_id: 2, slot: 1, port: 2, unit: 0, logical_port: 2, speed: 100G): DOWN

Message from syslogd@localhost at Dec 5 17:10:57 ...
stratum_bcm[3392]: INFO bcm_chassis_manager.cc:1755] State of SingletonPort (node_id: 1, port_id: 1, slot: 1, port: 1, unit: 0, logical_port: 1, speed: 100G): DOWN

Message from syslogd@localhost at Dec 5 17:10:57 ...
stratum_bcm[3392]: INFO bcm_chassis_manager.cc:1755] State of SingletonPort (node_id: 1, port_id: 2, slot: 1, port: 2, unit: 0, logical_port: 2, speed: 100G): DOWN
I1205 17:10:57.763020 3392 p4_service.cc:130] Pushing the saved forwarding pipeline configs read from /root/sdklt/config/pipeline_cfg.pb.txt...
I1205 17:10:57.763170 3394 gnmi_events.h:618] Handling N7stratum3hal25PortOperStateChangedEventE

Message from syslogd@localhost at Dec 5 17:10:57 ...
stratum_bcm[3392]: INFO p4_service.cc:130] Pushing the saved forwarding pipeline configs read from /root/sdklt/coI1205 17:10:57.763427 3394 gnmi_events.h:618] Handling N7stratum3hal25PortOperStateChangedEventE
W1205 17:10:57.763928 3392 p4_service.cc:151] Empty forwarding pipeline configs file: /root/sdklt/config/pipeline_cfg.pb.txt.

Message from syslogd@localhost at Dec 5 17:10:58 ...
stratum_bcm[3392]: INFO gnmi_events.h:618] Handling N7stratum3hal25PortOperStateChangedEventE
E1205 17:10:58.140655853 3392 server_chttp2.cc:40] {"created":"@1575565858.140509043","description":"Temporary failure in name resolution","errno":-3,"file":"external/com_github_grpc_grpc/src/core/lib/iomgr/resolve_address_posix.cc","file_line":108,"os_error":"Temporary failure in name resolution","syscall":"getaddrinfo","target_address":""}
E1205 17:10:58.141386 3392 hal.cc:231] StratumErrorSpace::ERR_INTERNAL: Failed to start Stratum external facing services. This is an internal error.
E1205 17:10:58.142772 3392 main.cc:137] Error when running Stratum HAL: Failed to start Stratum external facing services. This is an internal error.
E1205 17:10:58.143072 3392 bcm_chassis_manager.cc:149] Deleting BcmChassisManager while initialized_ is still true. You did not call Shutdown() before deleting the class instance. This can lead to unexpected behavior.
*** Aborted at 1575565858 (unix time) try "date -d @1575565858" if you are using GNU date ***
PC: @ 0x0 (unknown)
*** SIGABRT (@0xD40) received by PID 3392 (TID 0x7f6e7ae62700) from PID 3392; stack trace: ***
@ 0x7f6e898240e0 (unknown)
@ 0x7f6e883dbfff gsignal
@ 0x7f6e883dd42a abort
@ 0x7f6e8d2268e8 google::InitVLOG3__()
@ 0x7f6e8ba00fa4 stratum::hal::bcm::BcmPacketioManager::HandleKnetIntfPacketRx()
@ 0x7f6e8ba0288f stratum::hal::bcm::BcmPacketioManager::KnetIntfRxThreadFunc()
@ 0x7f6e8981a4a4 start_thread
@ 0x7f6e88491d0f clone
@ 0x0 (unknown)
Aborted

[pudelkoM]

I looked at the log again and found this:

E1205 17:10:58.140655853 3392 server_chttp2.cc:40] {"created":"@1575565858.140509043","description":"Temporary failure in name resolution","errno":-3,"file":"external/com_github_grpc_grpc/src/core/lib/iomgr/resolve_address_posix.cc","file_line":108,"os_error":"Temporary failure in name resolution","syscall":"getaddrinfo","target_address":""}

The empty target address is suspicious since you gave stratum this flag: -external_hercules_urls=0.0.0.0:28000. We did a rename before open source and this flag is now named -external_stratum_urls. There should be code to warn about empty URLs, but it was/is non functional at the moment. I created a fix (#59), once it's merged, please pull latest master and try again.

[pudelkoM]

The issue should be fixed now. Reopen this issue if not and create a new issue if you encounter more problems.