OSD Progress bar very late to update about iina

[saagarjha]

Aha:

Let me look into what is going on here.

[low-batt]

@svobs Is this the problem PR #4545 is addressing?

When I held the → key down the OSD it did update contrary to what the PR reports. The updates did lag behind, but would catch up once I let go of the key.

[svobs]

@svobs Is this the problem PR #4545 is addressing?

No, that is addressing the case where the user only does a single action to display the OSD, and the OSD is configured to stay up for more than a second, but despite the video continuing to play (playback time is increasing), the time displayed in the OSD would not change.

If this issue is referring to the tendency of the OSD to stop updating when the keyboard repeat is used, I filed a separate issue for that a couple of years ago - although I can't seem to find it now. I may have closed it. I did notice that the severity of that problem can change dramatically with each OS release + even minor code changes. I thought at one point that it had gone away for good, but I did recently notice that it was back in Sonoma.

[low-batt]

I had been testing under Ventura. I ran more tests today under Sonoma. I still can't reproduce this one. However if I change the → key binding to step by frame the OSD is showing pause/resume, which is supposed to be a hidden internal operation in that case, so that needs to be fixed.

[svobs]

When I held the → key down the OSD it did update contrary to what the PR reports. The updates did lag behind, but would catch up once I let go of the key.

Forgot to follow up with this. I investigated this problem much more deeply. It only happens when the key binding being repeated is bound a menu item. Bindings which are not attached to menu items do not have this problem. I found that the problem is not actually caused by a race condition or the mpv queue. I don't have 100% visibility but I'm very certain it's a consequence of Cocoa's event loop handling.

I think something is happening like this on the main thread (pseudocode):

// Begin DispatchQueue work item
while !keyEventQueue.isEmpty() {
  let keyEvent = keyEventQueue.popNext()
  if keyEvent.isMenuItemKeyEquivalent() {
    menu.handle(keyEvent)
    continue
  } else {
    sendToNextResponder(keyEvent)
    break
  }
}
// End DispatchQueue work item.
// DispatchQueue.main can now process next work item

In other words, the whole main thread seems to be getting blocked if menu item key equivalents come in. I believe this is done to give higher priority to menu items over any other events, and this an effect which is at the discretion of Apple's developers.

As I see it, there are two different ways to fix this. One is to override sendEvent() in AppDelegate and make sure that such loops don't happen, which could be tricky and risk having side effects. But an easier solution is to add function calls which explicitly redraw whatever needs to be drawn, such as the OSD and OSC, when receiving key events, and put them in MainWindow's methods and maybe a few other places.

[asgeirtj]

For what it's woth the problem at some point resolved itself. I played around using the mpv usd and when I changed back it was fixed.

[low-batt]

@asgeirtj All evidence is helpful. Thanks for reporting the latest developments.

I can not explain why switching to the mpv OSD and switching back to IINA's OSD would resolve the problem. The OSD settings merely control simple boolean flags. There isn't some additional data that could be corrected by disabling and reenabling IINA's OSD. Maybe @svobs can explain that behavior.

@svobs Some thoughts…

Redrawing upon receiving a key event seems really wrong. That is before a seek has finished, so the times may not have changed. This would only work in the case of repeated seeks as it would update the OSD to reflect the previous key press and seek. Or am I confused? In any case an application should not need to do this.

I've not been able to reproduce this on either of my machines. Feels like there is some contributing factor that has not been identified.

[svobs]

Redrawing upon receiving a key event seems really wrong. That is before a seek has finished, so the times may not have changed. This would only work in the case of repeated seeks as it would update the OSD to reflect the previous key press and seek. Or am I confused? In any case an application should not need to do this.

Doing so would not change the timing of any existing redraws. It would only be adding more redraws. Currently, the redraw process can get starved because each redraw is enqueued in a new DispatchQueue task separate from the key event processing task, and the key event processing must return in a timely manner for the redraws to be displayed.

I've not been able to reproduce this on either of my machines. Feels like there is some contributing factor that has not been identified.

It is still occurring for me on MacOS 14.3.1. Make sure your keyboard repeat rate is set to the highest value, and you are using a key binding which is associated with a menu item. For example, with IINA Default, I reproduced using the LEFT and RIGHT arrow keys.

[svobs]

Redrawing upon receiving a key event seems really wrong. That is before a seek has finished, so the times may not have changed.

Felt like I could add more to the explanation. Say the RIGHT arrow key is pressed. The following roughly happens (time flows downward):

Sequence #	main DQ	mpv DQ
1	processKeyEvent()
2	send seek cmd to mpv
3	...	(finish prev work items in queue)
4		Receive MPV_EVENT_SEEK from mpv
5		enqueue display of seek OSD msg on main queue
6	(finish previous work items in queue)	...
7	Display seek OSD msg

Steps 3 and 6 are symbolic, trying to highlight the dependency on the each queue due to being asynchronous. The problem occurs because Step 6 doesn't complete until the key events slow down and the previous work items are allowed to finish. At this point all of the previously enqueued "seek" OSD messages are executed in quick succession, which isn't as wasteful as it sounds because AppKit seems to have optimizations which cause excessive redraws to get dropped, but it isn't ideal.

On my fork I used a solution which uses a new, simple OSD queue (not a DispatchQueue) where work is added to in step 5. Then at the start of each key event, any work found in the OSD queue is executed and the queue is drained. I also modified the displayOSD function to do the same, so if it is able to execute instead of the key event, the OSD still gets displayed. This seems to work pretty well. (I also redraw the OSC with each key event because it can sometimes get starved as well, but that's outside the scope of this bug).

[low-batt]

This was the contributing factor I was missing:

Make sure your keyboard repeat rate is set to the highest value

In macOS System Settings on the Keyboard tab my Key repeat rate setting was one knotch short of the maximum value. When I set that to the maximum rate then the problem reproduces on both of my development machines. One is running macOS 13.6.3, the other macOS 14.3.

I confirmed your finding that this does not happen if the key is not a keyboard shortcut for a menu item.

And yes, if in IINA's settings on the UI tab in the On Screen Controller section the Auto hide after setting is set to 30 it is easy to see that the OSC exhibits the same problem with lack of updates.

I'm going to ask @saagarjha for guidance on this one.

@asgeirtj What setting are you using for Key repeat rate?

[asgeirtj]

@asgeirtj What setting are you using for Key repeat rate?

key repeat rate is in fastest slot. delay until repeat is shortest slot fwiw. I also disabled the hold aplhanumerical key for diactirics thing.

[saagarjha]

Need to look into this more but the gist of it is that the speculation is incorrect, mpv_set_wakeup_callback isn't being called at all during the repeated seek

[svobs]

Need to look into this more but the gist of it is that the speculation is incorrect, mpv_set_wakeup_callback isn't being called at all during the repeated seek

It's not that it's not getting called. It's being blocked. Seems to be due to previous DispatchQueue.main.sync calls when handling MPV_EVENT_SEEK and MPV_EVENT_PLAYBACK_RESTART.

You can unblock the queue in MPVController if you replace all the DispatchQueue.main.sync occurrences in the body of handleEvent with DispatchQueue.main.async. I suppose doing so is a necessary step for fixing this bug. But it still doesn't unblock the main DQ, so work piles up there (including the OSD redraws) until the key stops repeating.

[low-batt]

I instrumented IINA to show the queue behavior reported by @svobs. This first set of log messages shows holding the → key which goes through the menu. It shows the seek command being submitted but receiving the event and synchronizing the UI stalls until I lift the key.

Going Through Menu:

731475230732347 command seek
731475230734375 MPV_EVENT_SEEK
731475230734434 player.syncUI
731475231208340 command seek
731475231209113 MPV_EVENT_SEEK
731475231209178 player.syncUI
731475231268011 command seek
731475231321357 command seek
731475231378450 command seek
731475231431852 command seek
731475231484611 command seek
731475231537956 command seek
731475231591342 command seek
731475231644441 command seek
731475231697816 command seek
731475231750910 command seek
731475231804300 command seek
731475231857691 command seek
731475231911227 command seek
731475231963953 command seek
731475232015992 command seek
731475232068878 command seek
731475232121919 command seek
731475232174466 command seek
731475232227207 command seek
731475232279938 command seek
731475232332942 command seek
731475232386281 command seek
731475232439644 command seek
731475232496405 command seek
731475232553912 command seek
731475232607381 command seek
731475232661709 command seek
731475232715140 command seek
731475232769959 command seek
731475232823224 command seek
731475232878167 command seek
731475232930993 command seek
731475232983934 command seek
731475233036713 command seek
731475233089516 command seek
731475233141003 command seek
731475233193753 command seek
731475233246286 command seek
731475233299053 command seek
731475233352501 command seek
731475233406031 command seek
731475233459834 command seek
731475233512539 command seek
731475233566218 command seek
731475233622589 command seek
731475233678677 command seek
731475233731896 command seek
731475233787216 command seek
731475233828140 MPV_EVENT_SEEK
731475233829683 player.syncUI
731475233876645 MPV_EVENT_SEEK
731475233876695 player.syncUI
731475233878791 MPV_EVENT_SEEK
731475233878826 player.syncUI
731475233880436 MPV_EVENT_SEEK
731475233880475 player.syncUI
731475233882058 MPV_EVENT_SEEK
731475233882089 player.syncUI
731475233884200 MPV_EVENT_SEEK
731475233884229 player.syncUI
731475233884267 MPV_EVENT_SEEK
731475233884409 player.syncUI
731475233886169 MPV_EVENT_SEEK
731475233886194 player.syncUI
731475233886818 MPV_EVENT_SEEK
731475233886853 player.syncUI
731475233889159 MPV_EVENT_SEEK
731475233889188 player.syncUI
731475233890014 MPV_EVENT_SEEK
731475233890047 player.syncUI
731475233890676 MPV_EVENT_SEEK
731475233890702 player.syncUI
731475233890775 MPV_EVENT_SEEK
731475233890907 player.syncUI
731475233891692 MPV_EVENT_SEEK
731475233891730 player.syncUI
731475233892351 MPV_EVENT_SEEK
731475233892384 player.syncUI
731475233892938 MPV_EVENT_SEEK
731475233892974 player.syncUI
731475233893593 MPV_EVENT_SEEK
731475233893623 player.syncUI
731475233945435 MPV_EVENT_SEEK
731475233945527 player.syncUI
731475234068912 MPV_EVENT_SEEK
731475234071073 player.syncUI
731475234227086 MPV_EVENT_SEEK
731475234229830 player.syncUI
731475234270230 MPV_EVENT_SEEK
731475234270401 player.syncUI
731475234311826 MPV_EVENT_SEEK
731475234311991 player.syncUI
731475234395665 MPV_EVENT_SEEK
731475234395841 player.syncUI
731475234464360 MPV_EVENT_SEEK
731475234469723 player.syncUI
731475234545840 MPV_EVENT_SEEK
731475234548298 player.syncUI
731475234589304 MPV_EVENT_SEEK
731475234591765 player.syncUI
731475234671295 MPV_EVENT_SEEK
731475234673107 player.syncUI
731475234748123 MPV_EVENT_SEEK
731475234750719 player.syncUI
731475234750894 MPV_EVENT_SEEK
731475234751363 player.syncUI
731475234822294 MPV_EVENT_SEEK
731475234824139 player.syncUI
731475234908442 MPV_EVENT_SEEK
731475234911035 player.syncUI

This does not happen when I assign a key binding to seek that does not go through the menu.

Without Triggering Menu:

731475296476136 command seek 5
731475296476986 MPV_EVENT_SEEK
731475296477035 player.syncUI
731475296957423 command seek 5
731475296958220 MPV_EVENT_SEEK
731475296958260 player.syncUI
731475296989652 command seek 5
731475296995636 MPV_EVENT_SEEK
731475296995663 player.syncUI
731475297023081 command seek 5
731475297037123 MPV_EVENT_SEEK
731475297037163 player.syncUI
731475297057153 command seek 5
731475297078896 MPV_EVENT_SEEK
731475297078942 player.syncUI
731475297090700 command seek 5
731475297120689 MPV_EVENT_SEEK
731475297120760 player.syncUI
731475297131285 command seek 5
731475297158253 command seek 5
731475297162312 MPV_EVENT_SEEK
731475297162359 player.syncUI
731475297211995 command seek 5
731475297226175 command seek 5
731475297253164 MPV_EVENT_SEEK
731475297254895 player.syncUI
731475297258916 command seek 5
731475297280898 MPV_EVENT_SEEK
731475297282585 player.syncUI
731475297286184 MPV_EVENT_SEEK
731475297286213 player.syncUI
731475297316135 command seek 5
731475297319851 MPV_EVENT_SEEK
731475297319885 player.syncUI
731475297326195 command seek 5
731475297353555 MPV_EVENT_SEEK
731475297353585 player.syncUI
731475297359507 command seek 5
731475297386962 MPV_EVENT_SEEK
731475297387003 player.syncUI
731475297414503 command seek 5
731475297420152 MPV_EVENT_SEEK
731475297420176 player.syncUI
731475297425461 command seek 5
731475297458947 MPV_EVENT_SEEK
731475297458991 player.syncUI
731475297460923 command seek 5
731475297491398 MPV_EVENT_SEEK
731475297491429 player.syncUI
731475297494207 command seek 5
731475297524435 MPV_EVENT_SEEK
731475297524472 player.syncUI
731475297527661 command seek 5
731475297557702 MPV_EVENT_SEEK
731475297557740 player.syncUI
731475297563790 command seek 5
731475297590356 MPV_EVENT_SEEK
731475297590409 player.syncUI
731475297597035 command seek 5
731475297624538 MPV_EVENT_SEEK
731475297624571 player.syncUI
731475297633754 command seek 5
731475297657824 MPV_EVENT_SEEK
731475297657854 player.syncUI
731475297667497 command seek 5
731475297691039 MPV_EVENT_SEEK
731475297691067 player.syncUI
731475297700769 command seek 5
731475297724412 MPV_EVENT_SEEK
731475297724451 player.syncUI
731475297735630 command seek 5
731475297757731 MPV_EVENT_SEEK
731475297757764 player.syncUI
731475297772694 command seek 5
731475297790889 MPV_EVENT_SEEK
731475297790917 player.syncUI
731475297803086 command seek 5
731475297824390 MPV_EVENT_SEEK
731475297824422 player.syncUI
731475297838261 command seek 5
731475297857726 MPV_EVENT_SEEK
731475297857760 player.syncUI
731475297871363 command seek 5
731475297890536 MPV_EVENT_SEEK
731475297890566 player.syncUI
731475297904832 command seek 5
731475297924444 MPV_EVENT_SEEK
731475297924491 player.syncUI
731475297938079 command seek 5
731475297960757 MPV_EVENT_SEEK
731475297960780 player.syncUI
731475297972677 command seek 5
731475297991390 MPV_EVENT_SEEK
731475297991419 player.syncUI
731475298005212 command seek 5
731475298027526 MPV_EVENT_SEEK
731475298027559 player.syncUI
731475298039084 command seek 5
731475298057958 MPV_EVENT_SEEK
731475298057988 player.syncUI
731475298072579 command seek 5
731475298092247 MPV_EVENT_SEEK
731475298092273 player.syncUI
731475298105009 command seek 5
731475298124311 MPV_EVENT_SEEK
731475298124343 player.syncUI
731475298137739 command seek 5
731475298158916 MPV_EVENT_SEEK
731475298158950 player.syncUI
731475298171152 command seek 5
731475298190990 MPV_EVENT_SEEK
731475298191022 player.syncUI
731475298205018 command seek 5
731475298223965 MPV_EVENT_SEEK
731475298223997 player.syncUI
731475298238082 command seek 5
731475298256790 MPV_EVENT_SEEK
731475298256818 player.syncUI
731475298273227 command seek 5
731475298290575 MPV_EVENT_SEEK
731475298290607 player.syncUI
731475298306329 command seek 5
731475298321632 MPV_EVENT_SEEK
731475298321673 player.syncUI
731475298339672 command seek 5
731475298350202 MPV_EVENT_SEEK
731475298350232 player.syncUI
731475298371305 command seek 5
731475298382566 MPV_EVENT_SEEK
731475298382597 player.syncUI
731475298412975 command seek 5
731475298421564 MPV_EVENT_SEEK
731475298421596 player.syncUI
731475298440052 command seek 5
731475298454711 MPV_EVENT_SEEK
731475298454762 player.syncUI
731475298475693 command seek 5
731475298486317 MPV_EVENT_SEEK
731475298486343 player.syncUI
731475298515959 command seek 5
731475298525820 MPV_EVENT_SEEK
731475298525845 player.syncUI
731475298540930 command seek 5
731475298558409 MPV_EVENT_SEEK
731475298558446 player.syncUI
731475298574285 command seek 5
731475298599052 MPV_EVENT_SEEK
731475298599085 player.syncUI
731475298608327 command seek 5
731475298631915 MPV_EVENT_SEEK
731475298631947 player.syncUI
731475298643011 command seek 5
731475298670360 MPV_EVENT_SEEK
731475298670418 player.syncUI
731475298676339 command seek 5
731475298702902 MPV_EVENT_SEEK
731475298702932 player.syncUI
731475298713605 command seek 5
731475298739106 MPV_EVENT_SEEK
731475298739144 player.syncUI
731475298747075 command seek 5
731475298770209 MPV_EVENT_SEEK
731475298770240 player.syncUI
731475298780615 command seek 5
731475298802813 MPV_EVENT_SEEK
731475298802867 player.syncUI
731475298813892 command seek 5
731475298835995 MPV_EVENT_SEEK
731475298836041 player.syncUI
731475298847997 command seek 5
731475298869335 MPV_EVENT_SEEK
731475298869369 player.syncUI
731475298884242 command seek 5
731475298903105 MPV_EVENT_SEEK
731475298903132 player.syncUI
731475298917620 command seek 5
731475298937772 MPV_EVENT_SEEK
731475298937820 player.syncUI
731475298950387 command seek 5
731475298969576 MPV_EVENT_SEEK
731475298969606 player.syncUI
731475298983985 command seek 5
731475299002809 MPV_EVENT_SEEK
731475299002831 player.syncUI
731475299017230 command seek 5
731475299038899 MPV_EVENT_SEEK
731475299038931 player.syncUI
731475299050564 command seek 5
731475299069817 MPV_EVENT_SEEK
731475299069864 player.syncUI
731475299084294 command seek 5
731475299102598 MPV_EVENT_SEEK
731475299102628 player.syncUI
731475299116002 command seek 5
731475299136188 MPV_EVENT_SEEK
731475299136230 player.syncUI
731475299150509 command seek 5
731475299169472 MPV_EVENT_SEEK
731475299169510 player.syncUI
731475299184307 command seek 5
731475299207878 MPV_EVENT_SEEK
731475299207908 player.syncUI
731475299219016 command seek 5
731475299237307 MPV_EVENT_SEEK
731475299237336 player.syncUI
731475299252328 command seek 5
731475299273874 MPV_EVENT_SEEK
731475299273903 player.syncUI
731475299285817 command seek 5
731475299308674 MPV_EVENT_SEEK
731475299308704 player.syncUI
731475299319996 command seek 5
731475299341508 MPV_EVENT_SEEK
731475299341547 player.syncUI
731475299353850 command seek 5
731475299374796 MPV_EVENT_SEEK
731475299374826 player.syncUI
731475299387571 command seek 5
731475299405511 MPV_EVENT_SEEK
731475299405535 player.syncUI
731475299420156 command seek 5
731475299438843 MPV_EVENT_SEEK
731475299438891 player.syncUI
731475299453376 command seek 5
731475299471807 MPV_EVENT_SEEK
731475299471845 player.syncUI
731475299487701 command seek 5
731475299505584 MPV_EVENT_SEEK
731475299505615 player.syncUI
731475299522525 command seek 5
731475299534540 MPV_EVENT_SEEK
731475299534563 player.syncUI
731475299554098 command seek 5
731475299566937 MPV_EVENT_SEEK
731475299566975 player.syncUI
731475299588971 command seek 5
731475299599899 MPV_EVENT_SEEK
731475299599930 player.syncUI
731475299622222 command seek 5
731475299633033 MPV_EVENT_SEEK
731475299633063 player.syncUI
731475299657872 command seek 5
731475299667199 MPV_EVENT_SEEK
731475299667245 player.syncUI
731475299694563 command seek 5
731475299699001 MPV_EVENT_SEEK
731475299699030 player.syncUI
731475299724976 command seek 5
731475299731997 MPV_EVENT_SEEK
731475299732025 player.syncUI
731475299758598 command seek 5
731475299764295 MPV_EVENT_SEEK
731475299764321 player.syncUI
731475299792958 command seek 5
731475299798587 MPV_EVENT_SEEK
731475299798619 player.syncUI
731475299824071 command seek 5
731475299831870 MPV_EVENT_SEEK
731475299831905 player.syncUI
731475299858345 command seek 5
731475299864496 MPV_EVENT_SEEK
731475299864526 player.syncUI
731475299898967 command seek 5
731475299925169 command seek 5
731475299928369 MPV_EVENT_SEEK
731475299928465 player.syncUI
731475299939050 MPV_EVENT_SEEK
731475299939564 player.syncUI
731475299957489 command seek 5
731475299974898 MPV_EVENT_SEEK
731475299976120 player.syncUI
731475300002377 command seek 5
731475300006929 MPV_EVENT_SEEK
731475300008486 player.syncUI

On MPVController using sync instead of async when dispatching to the main queue that @svobs mentioned…

That has been changed in PR #4819 to fix data races. If somebody can think of a reason why sync is required please start a discussion in that PR.

Running on the PR and holding the → key now shows the MPVController queue is no longer blocked and IINA continues to read the MPV_EVENT_SEEK events. But the task queued to the main queue to call player.syncUI does not get a chance to run until I let up on the key.

Going Through Menu with Async Changes:

22:59:19.595 [iina][d] Method call trace:
731476750222352 command loadfile
731476753943651 command seek
731476753944455 MPV_EVENT_SEEK
731476753944494 player.syncUI
731476754418464 command seek
731476754419709 MPV_EVENT_SEEK
731476754419753 player.syncUI
731476754476736 command seek
731476754484511 MPV_EVENT_SEEK
731476754534331 player.syncUI
731476754563981 command seek
731476754564846 MPV_EVENT_SEEK
731476754617743 command seek
731476754618294 MPV_EVENT_SEEK
731476754671536 command seek
731476754675642 MPV_EVENT_SEEK
731476754726053 command seek
731476754727206 MPV_EVENT_SEEK
731476754779272 command seek
731476754782915 MPV_EVENT_SEEK
731476754834664 command seek
731476754835547 MPV_EVENT_SEEK
731476754887521 command seek
731476754888494 MPV_EVENT_SEEK
731476754940775 command seek
731476754952689 MPV_EVENT_SEEK
731476754993503 command seek
731476754999106 MPV_EVENT_SEEK
731476755047115 command seek
731476755048748 MPV_EVENT_SEEK
731476755099945 command seek
731476755100244 MPV_EVENT_SEEK
731476755153238 command seek
731476755153717 MPV_EVENT_SEEK
731476755206328 command seek
731476755206770 MPV_EVENT_SEEK
731476755258562 command seek
731476755259027 MPV_EVENT_SEEK
731476755312069 command seek
731476755312625 MPV_EVENT_SEEK
731476755365671 command seek
731476755366120 MPV_EVENT_SEEK
731476755419257 command seek
731476755422536 MPV_EVENT_SEEK
731476755472342 command seek
731476755473636 MPV_EVENT_SEEK
731476755525122 command seek
731476755525453 MPV_EVENT_SEEK
731476755577948 command seek
731476755578376 MPV_EVENT_SEEK
731476755630813 command seek
731476755631228 MPV_EVENT_SEEK
731476755683040 command seek
731476755683465 MPV_EVENT_SEEK
731476755735856 command seek
731476755736339 MPV_EVENT_SEEK
731476755790530 command seek
731476755791006 MPV_EVENT_SEEK
731476755843308 command seek
731476755849332 MPV_EVENT_SEEK
731476755896784 command seek
731476755898473 MPV_EVENT_SEEK
731476755950414 command seek
731476755950870 MPV_EVENT_SEEK
731476756003692 command seek
731476756004290 MPV_EVENT_SEEK
731476756057581 command seek
731476756058202 MPV_EVENT_SEEK
731476756111133 command seek
731476756116585 MPV_EVENT_SEEK
731476756169780 command seek
731476756170174 MPV_EVENT_SEEK
731476756223027 command seek
731476756225254 MPV_EVENT_SEEK
731476756275493 command seek
731476756281884 MPV_EVENT_SEEK
731476756328524 command seek
731476756333657 MPV_EVENT_SEEK
731476756382605 command seek
731476756383659 MPV_EVENT_SEEK
731476756436052 command seek
731476756436613 MPV_EVENT_SEEK
731476756489188 command seek
731476756490230 MPV_EVENT_SEEK
731476756546272 command seek
731476756546704 MPV_EVENT_SEEK
731476756600193 command seek
731476756602327 MPV_EVENT_SEEK
731476756652776 command seek
731476756656250 MPV_EVENT_SEEK
731476756709991 command seek
731476756710620 MPV_EVENT_SEEK
731476756763097 command seek
731476756763469 MPV_EVENT_SEEK
731476756815907 command seek
731476756816276 MPV_EVENT_SEEK
731476756869223 command seek
731476756869806 MPV_EVENT_SEEK
731476756921600 command seek
731476756922977 MPV_EVENT_SEEK
731476756976285 command seek
731476756977524 MPV_EVENT_SEEK
731476757028915 command seek
731476757029330 MPV_EVENT_SEEK
731476757083475 command seek
731476757083966 MPV_EVENT_SEEK
731476757136589 command seek
731476757137645 MPV_EVENT_SEEK
731476757191069 command seek
731476757191451 MPV_EVENT_SEEK
731476757244002 command seek
731476757244431 MPV_EVENT_SEEK
731476757296597 command seek
731476757297056 MPV_EVENT_SEEK
731476757350764 command seek
731476757351237 MPV_EVENT_SEEK
731476757404416 command seek
731476757404839 MPV_EVENT_SEEK
731476757457603 command seek
731476757458436 MPV_EVENT_SEEK
731476757509656 command seek
731476757510477 MPV_EVENT_SEEK
731476757551756 player.syncUI
731476757553644 player.syncUI
731476757597411 player.syncUI
731476757597488 player.syncUI
731476757597986 player.syncUI
731476757598077 player.syncUI
731476757600563 player.syncUI
731476757600660 player.syncUI
731476757600704 player.syncUI
731476757600753 player.syncUI
731476757602039 player.syncUI
731476757603259 player.syncUI
731476757603842 player.syncUI
731476757604421 player.syncUI
731476757605564 player.syncUI
731476757605961 player.syncUI
731476757606305 player.syncUI
731476757606356 player.syncUI
731476757607517 player.syncUI
731476757608581 player.syncUI
731476757609626 player.syncUI
731476757660820 player.syncUI
731476757782195 player.syncUI
731476757859770 player.syncUI
731476757860022 player.syncUI
731476757943637 player.syncUI
731476758019940 player.syncUI
731476758099539 player.syncUI
731476758105969 player.syncUI
731476758182844 player.syncUI
731476758225067 player.syncUI
731476758301965 player.syncUI
731476758396760 player.syncUI
731476758397146 player.syncUI
731476758397508 player.syncUI
731476758461432 player.syncUI
731476758542014 player.syncUI
731476758583380 player.syncUI
731476758627317 player.syncUI
731476758706141 player.syncUI
731476758746597 player.syncUI
731476758821233 player.syncUI
731476758903166 player.syncUI
731476758946305 player.syncUI
731476759025388 player.syncUI
731476759032073 player.syncUI
731476759102684 player.syncUI
731476759146378 player.syncUI
731476759223751 player.syncUI
731476759269634 player.syncUI
731476759345790 player.syncUI
731476759382610 player.syncUI
731476759385396 player.syncUI
731476759461981 player.syncUI
731476759544691 player.syncUI
731476759589040 player.syncUI

This is some of the code I used to generate those messages.

MethodTracer::

class MethodTracer {

  struct record {
    var time: Date = Date()
    var method: String
  }

  private static let baseTime: Date = Date()

  @Atomic private static var records: [record] = []

  static func recordCall(_ method: String = #function) {
    $records.withLock { $0.append(record(method: method)) }
  }

  static func log() {
    $records.withLock {
      var lines: String = ""
      for entry in $0 {
        if !lines.isEmpty {
          lines += "\n"
        }
        lines.append("\(Int(entry.time.timeIntervalSinceReferenceDate * 1000000)) \(entry.method)")
      }
      Logger.log("Method call trace:\n" + lines)
    }
  }
}

[low-batt]

Starting with the Xcode application template I added a menu item with → as a shortcut key along with the following custom NSWindow class:

class Window: NSWindow {

  private var counter = 0

  private func test(_ invoker: String) {
    counter += 1
    let count = counter
    print("\(count) \(invoker)")
    DispatchQueue.main.async {
      print("\(count) Task Run")
    }
  }

  override func keyDown(with event: NSEvent) { test("Key") }

  @IBAction func menuItem(_ sender: AnyObject) { test("Menu") }
}

This test program reproduces the behavior seen. When the macOS Key repeat rate setting is set to the maximum rate, holding down the shortcut key for a menu item blocks other tasks from running on the main dispatch queue. This does not occur when a key not associated with a menu is held down. This is the output from that test run:

Xcode Console::

1 Menu
1 Task Run
2 Menu
2 Task Run
3 Menu
3 Task Run
4 Menu
5 Menu
6 Menu
7 Menu
8 Menu
9 Menu
10 Menu
11 Menu
12 Menu
13 Menu
14 Menu
15 Menu
16 Menu
17 Menu
18 Menu
19 Menu
20 Menu
21 Menu
22 Menu
23 Menu
24 Menu
25 Menu
26 Menu
27 Menu
28 Menu
29 Menu
4 Task Run
5 Task Run
6 Task Run
7 Task Run
8 Task Run
9 Task Run
10 Task Run
11 Task Run
12 Task Run
13 Task Run
14 Task Run
15 Task Run
16 Task Run
17 Task Run
18 Task Run
19 Task Run
20 Task Run
21 Task Run
22 Task Run
23 Task Run
24 Task Run
25 Task Run
26 Task Run
27 Task Run
28 Task Run
29 Task Run

30 Key
30 Task Run
31 Key
31 Task Run
32 Key
32 Task Run
33 Key
33 Task Run
34 Key
34 Task Run
35 Key
35 Task Run
36 Key
36 Task Run
37 Key
37 Task Run
38 Key
38 Task Run
39 Key
39 Task Run
40 Key
40 Task Run
41 Key
41 Task Run
42 Key
42 Task Run
43 Key
43 Task Run
44 Key
44 Task Run
45 Key
45 Task Run
46 Key
46 Task Run
47 Key
47 Task Run
48 Key
48 Task Run
49 Key
49 Task Run
50 Key
50 Task Run
51 Key
51 Task Run
52 Key
52 Task Run
53 Key
53 Task Run
54 Key
54 Task Run
55 Key
55 Task Run
56 Key
56 Task Run
57 Key
57 Task Run
58 Key
58 Task Run
59 Key
59 Task Run
60 Key
60 Task Run
61 Key
61 Task Run
62 Key
62 Task Run
63 Key
63 Task Run
64 Key
64 Task Run
65 Key
65 Task Run
66 Key
66 Task Run
67 Key
67 Task Run
68 Key
68 Task Run
69 Key
69 Task Run
70 Key
70 Task Run
71 Key
71 Task Run
72 Key
72 Task Run
73 Key
73 Task Run
74 Key
74 Task Run

If I reduce the Key repeat rate once notch down from the maximum the tasks get a chance to run.

The full test program: Starving.tar.gz

[low-batt]

I took the test program to a MacBookAir8,2 running macOS Catalina 10.15.7 and the problem reproduced on that Mac.

This confirms this task scheduling behavior is not a recent change.

[saagarjha]

Ok so as expected NSMENU_IS_THROTTLING_REPEATED_MENU_ITEM_INVOCATIONS basically just sleeps to slow down menu events to at most once every 25ms (perhaps to protect the highlighting code?). Unfortunately the way this is implemented means that during this throttling it basically blocks the main thread because it waits, a new event comes in, and then it waits again, so nothing else can run. I don't see a way to opt out of this behavior but I wonder if we can ask for it to be removed? It seems counterproductive for us.

[low-batt]

Really not good to be sleeping while tasks are waiting. If possible an improvement would be to delay the event by running a task from the dispatch queue, only sleeping if there are no tasks to run. Or allow the application to opt out of this event rate throttling. Did you want to send Apple feedback, or do you want me to do it?

I tried this code as an experiment:

    NSEvent.addLocalMonitorForEvents(matching: .keyDown) { [self] event in
      guard event.keyCode == 124, event.isARepeat else {
        lastInvocation = event.timestamp
        return event
      }
      let timeSinceLast = event.timestamp - lastInvocation
      guard timeSinceLast < minTimeBetween else {
        lastInvocation = event.timestamp
        return event
      }
      print(String(format: "Time since last key event (%.1f ms) is too short, dropping event", timeSinceLast * 1000))
      return nil
    }

I found I had to drop an event if the time between events was less than 70 ms. Dropping events allowed the main queue tasks to run.

What should IINA do about this?

• Nothing, consider this expected macOS behavior
• Force drawing in the menu action
• Throttle event rate by dropping events

I'm thinking maybe nothing as users can add key bindings that are not tied to the menu. If you are really interested in high speed scrubbing, a key that is not a menu shortcut will perform better.

Thoughts?

[svobs]

Would definitely be great if Apple would improve this. Supposedly in Sonoma the menu system was "completely rewritten to take advantage of Cocoa" - what was it before? Some ancient NeXTStep-Apple Frankenstein stuff from 30 years ago? Its functionality has barely changed since Mac OS 10.0. Personally I've barely noticed much difference in menu functionality at all (erm, other than those _XMIGPostNotification crashes which were everywhere for a while...). The behavior at issue goes way back too...

Just wanted to point out the other "option" to work around this. You can subclass sendEvent inNSApplication to intercept and handle the key events using homegrown code, thus bypassing the menu item events and their lazy solution to rate limiting. If you're curious, I dug up some old code and got it updated and working which shows how to do this. Take a look at this branch and in particular the IINNApplication class. It actually sends to the main menu right away currently, but could be easily modified to process key events internally instead.

Seems like such a solution has been the standard way to build games and other "nonstandard" Mac apps in the past. But once you start messing with event handling priorities you open the door to plenty of possible side effects and corner cases. It's useful to ask just how useful it would be to repeat key events more than... every 25ms? Which ≈ 40 events per second. Seems like that should be pretty good for most everything? Unless someone comes up with a Lua script which can play Doom 😉

[saagarjha]

Filed as FB13685409

[low-batt]

Hopefully Apple will reply to FB13685409. I'm still waiting for Apple to reply to two I entered back in January 2023. I'm hoping we will quickly hear from Apple as to what the offending code is trying to accomplish.

Looking like Apple did not fully fix the _XMIGPostNotification crashes. Firefox has a report of a crash under macOS 14.3.1.

So IINAApplication is merely an example, that branch does not contain a fix for this issue, yes?

[saagarjha]

See also https://twitter.com/philzet/status/1767964426538512625

[svobs]

Looking like Apple did not fully fix the _XMIGPostNotification crashes. Firefox has a report of a crash under macOS 14.3.1.

I saw that. Not sure it's exactly the same bug, but definitely seems like there are a few loose bolts still.

It's too bad Apple didn't take the opportunity to start replacing AppKit with something more modern. They can't keep deferring it much longer. It's taken me the last 2 years to figure out how to build basic UI features for Cocoa. The same kinds of things I've been able to learn in weeks using modern webapp libraries. And it's still challenging to work with because it's fundamentally really old tech which existed before anyone knew how to organize UI, and thus has poor separation of concerns and requires perfect developer discipline. I imagine Apple would be having trouble retaining people on the Mac teams. Most developers wouldn't want to keep working with this stuff.

So IINAApplication is merely an example, that branch does not contain a fix for this issue, yes?

It does not, unfortunately. If I remember right, once I started going down that path there were many corner cases which popped up. For example, you'd need to detect when open file dialogs and editors in text fields have focus. Because the built-in responder chain does provides the logic to figure out which controls should receive key events, so any code which preempts that mechanism has to assume that responsibility. Which is just a little bit too much of a headache...

I still think the best approach is just to force OSD/OSC redraws from the menu item handler. It doesn't appear that Apple's sleeps are actually limiting video playback (empirically...not sure how to write a test to quantify this) since OpenGL does not use the main DQ. So having ~40 redraws per sec of these components via explicit draws should be fine.