CGZIP: genwqe_maint doesn't start when card is connected about genwqe-user

[fhaverkamp]

Only a little complicated way to figure out which cxl device is really a CGZIP one:

haver@tul2eth3:~/genwqe/genwqe-user$ lspci | grep accel
0002:01:00.0 Processing accelerators: IBM Device 0477 (rev 01)
0004:01:00.0 Processing accelerators: IBM Device 0477 (rev 01)
0007:00:00.0 Processing accelerators: IBM Device 0602 (rev 01)
0008:00:00.0 Processing accelerators: IBM Device 0602 (rev 01)

The 0602 is a CGZIP device, but that belongs to the "mother" device with the cxl device id 0744.
So which 0602 belongs now to which 0744 device to solve the question which "device" number to take?

One way:
haver@tul2eth3:/sys/bus/pci/devices**/0002:01:00.0**$ ls
broken_parity_status dma_mask_bits enable numa_node resource subsystem_device
class driver iommu_group of_node resource0 subsystem_vendor
config driver_override irq pci0007:00 resource0_wc uevent
consistent_dma_mask_bits eeh_config_addr local_cpulist power resource2 vendor
cxl eeh_mode local_cpus remove resource2_wc vpd
device eeh_pe_config_addr modalias rescan resource4
devspec eeh_pe_state msi_bus reset subsystem

So pci0007:00 belongs to the 0002:01:00.0 device. A way back from the 0007 device to the 02:01:00 device I was unable to find.

haver@tul2eth3:/sys/bus/pci/devices/0004:01:00.0$ ll cxl/
total 0
drwxr-xr-x 3 root root 0 Apr 5 15:19 ./
drwxr-xr-x 5 root root 0 Apr 5 15:18 ../
drwxr-xr-x 4 root root 0 Mar 22 16:01 card1/
haver@tul2eth3:/sys/bus/pci/devices/0004:01:00.0$

And here the link from the 4:01:00 device to card1. Also not so easily to be found.

But now the tricky thing: We need to consider that there are multiple non CGZIP cards in the system, so not every 0477 device has really a 0602 device. There could be 0477 devices with none or different virtual devices.

[fhaverkamp]

I wonder ... if we can have udev wait for general capi cards to appear
and if those have a pciXXXX link in their sysfs tree, we follow that link
and if we see the subdevice being a 0603 we know it is a CGZIP and we can start the daemon.
maybe that works.

Any better ideas are welcome.

[fhaverkamp]

Manish found a good way:

haver@tul2eth3:$ cat /sys/class/cxl/afu1.0s/device/cr0/device
0x0602
haver@tul2eth3:$ cat /sys/class/cxl/afu0.0s/device/cr0/device
0x0602
haver@tul2eth3:~$

Gabriel, can we use that? If we see the 0602 in this sysfs entry, the associated afu is a CGZIP one and we could start the maintenance daemon for this card.

[fhaverkamp]

@ibm-genwqe Can we close that? Will we ever fix this?

[fhaverkamp]

I think we might not fix that. Please reopen if there is demand for this.