Freezes when using more than one async prompt handler about ohmyzsh

[carlosala]

Hey! I can't reproduce. Which theme are you using?

[hwti]

ZSH_THEME="muse"
COMPLETION_WAITING_DOTS="true"
plugins=(git)

I don't expect the git plugin or the theme to have any effect, as I can reproduce by registering handlers in ~/.zshrc (and I don't need to be in a git repository to see the freezes).

I just discovered that for terminal panels opened in Kate or Dolphin, it's even worse than Konsole / VT !
In these applications, even one _omz_register_handler is enough to freeze the terminal for a few seconds on launch.
I still need at least two to reproduce the issue when pressing enter quickly.

[hwti]

With the two empty _omz_register_handler in my example, I see :

• on each prompt, /usr/bin/true and /usr/bin/cat are executed
• when the shell is frozen, it is reading on the read end of a pipe connected to the stdout of cat

With set -x, the last line is +_omz_async_callback:15> _OMZ_ASYNC_OUTPUT[$handler]=+_omz_async_callback:15> cat .
So it's the _OMZ_ASYNC_OUTPUT[$handler]="$(cat <&$fd)".
With echo aaa in function foo(), I see with set -x that when I get the issue the +foo:2> echo aaa doesn't appear : it looks like the handler process is blocked.
Replacing (exit $ret) by () { return $ret } seems to fix the issue when using the empty handlers at least (and it is faster).
I don't know why the subshell would block (so delay the handler) when I have the issue.

But I don't understand how the async_prompt.zsh is supposed to work.
builtin echo $handler triggers _omz_async_callback : it reads handler, and then run the cat <&$fd which waits for the handler process to exit (or close stdout).
With a slow handler (for example sleep 2), I clearly see the shell blocking on almost all prompts, so it doesn't seem to be asynchronous at all !
Without this issue maybe the (exit $ret) could still cause problems, but it would be a delay for the async handler, not the shell being frozen.
Ideally the callback would only run on HUP (when the handler process exits), but zle -F doesn't allow to select what we want.

Btw, the two exec could probably be avoided :

• the $(cat <&$fd) could be replaced by something like read -r -d '' -u $fd.
• It seems the command true is no longer needed on zsh >= 5.8 : zsh-users/zsh-autosuggestions#612

[mcornella]

Hi Loïc, thanks for the thorough report and the improvement pointers!

I've been testing out your reproduction and I can see the same issue. It looks like zle is calling the _omz_async_callback function even though the fd has no data ready to read, which is not what we intended to happen.

For example, with this modification

 # Called when new data is ready to be read from the pipe
 function _omz_async_callback() {
   emulate -L zsh
+  zmodload zsh/zselect
 
   local fd=$1   # First arg will be fd ready for reading
   local err=$2  # Second arg will be passed in case of error
@@ -116,18 +113,23 @@ function _omz_async_callback() {
   if [[ -z "$err" || "$err" == "hup" ]]; then
     # Get handler name from first line
     local handler
-    read handler <&$fd
+    read -u $fd handler
 
     # Store old output which is supposed to be already printed
     local old_output="${_OMZ_ASYNC_OUTPUT[$handler]}"
 
     # Read output from fd
-    _OMZ_ASYNC_OUTPUT[$handler]="$(cat <&$fd)"
-
-    # Repaint prompt if output has changed
-    if [[ "$old_output" != "${_OMZ_ASYNC_OUTPUT[$handler]}" ]]; then
-      zle reset-prompt
-      zle -R
+    if ! zselect -t 0 -r $fd; then
+      zle -M "$fd ($handler): no data to read
+$(ls -l /proc/$$/fd)"
+    else
+      read -u $fd -r -d '' "_OMZ_ASYNC_OUTPUT[$handler]"
+
+      # Repaint prompt if output has changed
+      if [[ "$old_output" != "${_OMZ_ASYNC_OUTPUT[$handler]}" ]]; then
+        zle reset-prompt
+        zle -R
+      fi
     fi
 
     # Close the fd

I can see printed that the pipe does not yet have any data to read, even though it is readable (and therefore zle calls the callback):

➜  ohmyzsh git:(master) ✗
12 (foo): no data to read

We'll have to research this further and find the proper way to go about this before a full release. Thanks again for the report!

[hwti]

I can see printed that the pipe does not yet have any data to read, even though it is readable (and therefore zle calls the callback):

It has data to read when _omz_async_callback is called : the handler name (read by read handler <&$fd).

In my PR, I stored the handler name in an associative array (to get it back from the fd), instead of writing it to the pipe.
The PID is still written, but it is read before setting the callback, so now _omz_async_callback is only called when the completion function writes its output (ideally right at the end).

I still don't know what happens with the subshell (which created the full shell freeze, even with empty async handlers, when combined with the wait in _omz_async_callback).
Most of the time it only delays the prompt update by about 10-20ms (compared to using a function).
But on the first prompt (in Dolphin / Kate especially), I can still sometimes see a 10s delay for the first handler, and a 18-20s delay for the second one (and if I press Ctrl+D, the terminal panel only closes after the delay).
That's why my PR is still in draft : replacing (exit $ret) by () { return $ret } is faster and avoids the issue, but I would like to understand why (especially since async handler might to use subshells).

I will create separate PRs for the optimization (true / cat) and to make git_prompt_status asynchronous.

[mcornella]

It has data to read when _omz_async_callback is called : the handler name (read by read handler <&$fd).

That's such an obvious mistake 😅

For now, I'm going to un-draft #12304 (and make a small improvement), and since you are still noticing delays, let's open a second issue for that with a reproducible build, as well as maybe a recording of the issue whilst tracing the functions with functions -t <function_name>.

[hwti]

For now, I'm going to un-draft #12304 (and make a small improvement), and since you are still noticing delays, let's open a second issue for that with a reproducible build, as well as maybe a recording of the issue whilst tracing the functions with functions -t <function_name>.

I don't see any delay with the PR.
Since I replaced the subshell by a function (which should always be better anyway), the issue is fixed, but I don't know why it happened in the first place.

I'm currently daily-driving the PR, with also two other commits (avoiding true/cat, and making git_prompt_status async), and everything seems to work perfectly.

[mcornella]

Oh nice, then that's going to be the end of the issue. I just pushed 2 improvements, I'll test in the oldest versions we support and if all's good I'll merge it.