19.x release has long delay in unbinding the Nvidia GPU about system76-power

[flukejones]

I have tried to see if I can isolate a commit where the issue appears by checking out and building historical commits. This seems to be a dead-end. I'll now try debugging and see what I can do here.

I suspect the issue may be with Ubuntu 19.04. Can anyone confirm latest build from master works fine on 18.x?

[mmstick]

It works fine in 18.x, yes. The latest build is actually what you get in Pop!_OS 18.04 and 18.10 stable. Our GitHub repo is monitored by a packaging CI which automatically packages the master branches of each repo with a debian directory to Pop!_OS Proposed, which soon gets swept up into stable and released into a new ISO. Code doesn't get to master if it hasn't already had a code review and been tested in the lab by QA, as our packaging CI also builds each branch into a new temporary repository.

[flukejones]

Ah okay, good to know 😺 . Have there been internal tests for 19.04? I'm curious to know if others are having the same issue. It certainly seems related to 19.x now, but I'm unsure quite where to start in debugging.

[mmstick]

Yes, see https://github.com/orgs/pop-os/projects/8

This is causing switchable graphics issues in 19.04.

[flukejones]

Thanks for the link! I'll see if I can help out here. The delay also means that shutdown/standby/reboot etc are delayed until the unbind is complete, so there may be other things affected?

[flukejones]

Issue might be in sysfs-class around the file_write. Something seems to be making this take a long time to complete, unsure if sysfs-class is the actual issue, or if something in 19.04 is causing the write to be delayed - but this is the area I'll be looking in for now.

Apr 16 17:24:58 pop-os system76-power[1025]: [INFO] Disabling graphics power
Apr 16 17:24:58 pop-os system76-power[1025]: [INFO] xhci_hcd: Unbinding 0000:01:00.2
Apr 16 17:24:58 pop-os system76-power[1025]: [INFO] unbinding took 1.317832ms
Apr 16 17:24:58 pop-os system76-power[1025]: [INFO] nvidia-gpu: Unbinding 0000:01:00.3
Apr 16 17:28:33 pop-os system76-power[1025]: [INFO] unbinding took 3.58m
Apr 16 17:28:33 pop-os system76-power[1025]: [INFO] snd_hda_intel: Unbinding 0000:01:00.1
Apr 16 17:28:33 pop-os system76-power[1025]: [INFO] unbinding took 191.073818ms
Apr 16 17:28:33 pop-os system76-power[1025]: [INFO] unbind function took 3.58m

[mmstick]

This would be a kernel issue, but we could perhaps send all I/O tasks to a background thread to prevent the daemon from stalling.

[flukejones]

Confirming it's a kernel issue. I tried:

• 4.18.0, no issue, unbinds quick
• 5.0.0, unbind takes 3 minutes
• 5.0.7, unbind takes 3 minutes

I haven't tried 5.1 RC kernels yet. Probably using a threaded model would be best in the long run, or even futures - but I can't help with this as I have limited time (and need to earn a living) sorry.

[jackpot51]

I found that the i2c-nvidia-gpu module is the cause, it was recently added to Linux. I am blacklisting it by default on the oryp5, where this problem occurs.

[flukejones]

@jackpot51 excellent! Thank you for the information and fix!