backspace doesn't work in 1.9 sometimes about rb-readline

[rdp]

hmm I don't see any option to upload a file, so here it is.

From ebcf6a4aed1cc4aece1225098d3f8ac0d7c540d2 Mon Sep 17 00:00:00 2001
From: rogerdpack 
Date: Fri, 27 May 2011 16:13:37 -0600
Subject: [PATCH] reset output handle when detected stale

---
 lib/rbreadline.rb |    6 +++++-
 1 files changed, 5 insertions(+), 1 deletions(-)

diff --git a/lib/rbreadline.rb b/lib/rbreadline.rb
index c7043fa..56bbf88 100644
--- a/lib/rbreadline.rb
+++ b/lib/rbreadline.rb
@@ -1545,7 +1545,11 @@ module RbReadline
          @GetConsoleScreenBufferInfo.Call(@hConsoleHandle,csbi)
          x,y = csbi[4,4].unpack('SS')
          x = dpos
-         @SetConsoleCursorPosition.Call(@hConsoleHandle,y*65536+x)
+         ret = @SetConsoleCursorPosition.Call(@hConsoleHandle,y*65536+x)
+         if ret == 0
+          @hConsoleHandle = @GetStdHandle.Call(STD_OUTPUT_HANDLE)
+         end
+
          @_rl_last_c_pos = dpos
          return
       end
-- 
1.7.4.msysgit.0

[luislavena]

Hey @rdp (Roger)

Why @hConsoleHandle is not valid? If is an instance variable shouldn't run out of scope or get garbage collected, so why you need to retrieve the handle again?

[Spakman]

@rdp Nice one with the report and fix (I haven't yet applied and tested the patch).

Is there any way to write a test for the bug; perhaps by testing something in a loop? I know it's intermittent, so that might not be easy. It might be a case where writing the test is not possible or just not worth the effort.

[rdp]

@luislavena I didn't get that far in my investigation. After I finding a work-around after a couple of hours I was too tired to look much further :)

A few hints: it's only 1.9, never 1.8, when I tried "setting it several times to start with" it always returned 23 for the handle, but then when I reset it [this patch] it returned 15, and then worked after that. It might be an ffi bug--possible I don't know. Somehow during the course of ruby loading it switches from returning a "bad" value to returning a "good" one. I suppose I could investigate further if you'd like me too.

@Spakman the test for it would be "if I hit backspace, does it move my cursor back by a space or not" (like when running it within a real windows console). Unfortunately like I mentioned I haven't found a reliable way to actually reproduce it failing like that [I mean it does re-occur every so often, though I'm not sure what triggers it reliably]. So anyway in your environments that test would probably always pass, but in mine every so often, the test would actually be useful. This is a tough one to automate the test on, unless I can figure out a reliable way to reproduce it at all.
Thanks!
-roger-

[Spakman]

Are we going to include this in 0.4.1 or wait to test it some more and aim to get it in 0.5.0?

[luislavena]

@Spakman: I think the background issue is other, so will rather skip this from 0.4.1

[cfis]

Some more info on this.

Ruby 1.9.3, built with MSV++ 2010 (experimenting).

With a dos window, all works well. With console2 though, the backspace key doesn't work (cursor doesn't more). With Roger's patch, the first time you hit backspace the cursor actually moves left one space but does not delete the underlying character. From then on, every time you hit backspace it works fine.

Using a mingw build, I have yet to encounter the problem.

What's interesting, is the handle always opens up as 23 and then 15 (so what roger saw).

And then to make it even more confusing, it now just started working always....weird.

[luislavena]

@cfis:

Using a mingw build, I have yet to encounter the problem.

I've encountered this issue randomly several times with 1.9.2 and 1.9.3dev too.

This is getting really tricky to debug...

[cfis]

And today I saw it again using console + cmd, but things are ok with console + bash (from msys).

Charlie

[Spakman]

Are you guys experiencing the bug with @rdp's patch applied?

[cfis]

rdp's patch fixes only after it breaks once (see my note above).

[luislavena]

Ok, I think I found a repro, and the cause seems to be Console2 corrupt Std handle.

After backspace stop working, RbReadline.instance_variable_get(:"@hConsoleHandle") return 27, on a normal Console2 or cmd.exe, the value is 7

Now, the following DL example: https://gist.github.com/1175823 reports the following:

Working cmd.exe:

In: 3
Out: 7
Err: 11

On working Console2:

In: 3
Out: 7
Err: 11

On broken Console2:

In: 7
Out: 23
Err: 11

And obtain similar values using this pure C version: https://gist.github.com/1175831

Now, after the corruption, any new tab inside the same session of Console2 fails, but starting a new Console2 application works.

Can you guys use those when the backspace issue happen again on your system? Would like to verify we all get the same results.

Thank you.

[cfis]

Hi Luis,

I'm not sure what you are asking us to do? Sorry if I'm being dense.

Charlie

[luislavena]

Sorry for not explaining myself, was really late when I posted that :-P

The next time you guys encounter this issue, can you guys execute the pure-ruby and the C version in the same console the issue is happening? Also test on a new cmd.exe and another instance of Console2

Want to discard that is Ruby (or rb-readline) the one causing this.

[luislavena]

So, the news I found, using both the Ruby and C version of the GetStdHandle code, I can confirm:

ruby 1.9.3dev (2011-08-27 revision 33101) [i386-mingw32]

Broken Console2:

C:\Users\Luis\Projects\sandbox\console>ruby status.rb
In: 7
Out: 23
Err: 11

C:\Users\Luis\Projects\sandbox\console>gcc status.c -o status.exe

C:\Users\Luis\Projects\sandbox\console>status
In: 7
Out: 23
Err: 11

New Tab inside same Console2:

C:\Users\Luis\Projects\sandbox\console>ruby status.rb
In: 3
Out: 7
Err: 11

C:\Users\Luis\Projects\sandbox\console>status
In: 3
Out: 7
Err: 11

New Console2:

C:\Users\Luis\Projects\sandbox\console>ruby status.rb
In: 3
Out: 7
Err: 11

C:\Users\Luis\Projects\sandbox\console>status
In: 3
Out: 7
Err: 11

Once that current console is broken, it fails systematically and requires a new cmd.exe, either by starting a new console2 or by starting a new tab inside the same process.

I'm going to see if I can reproduce this using cmd.exe alone. If doesn't fail, then the issue can be narrowed to Console2.

[cfis]

Ok, I have 3 tabs in console2 - cmd, bash, vc cmd (runs vcvars.bat). In all cases backspace works at the command prompt, but when I type in IRB, backspace is broken in all 3. Exit IRB, all is well again. Note I used ruby status.rb in all cases.

Using cmd:

In: 3
Out: 7
Err: 11

Using Bash:

In: 11
Out: 23
Err: 27

vc cmd:

In: 3
Out: 11
Err: 23

Now, return the experiment using a regular dos window (no console2):

In: 3
Out: 7
Err: 11

Same as console2, IRB also broken. I think this rules out Console2 as the culprit, at least for on my machine. Note that I recompiled from ruby 1_9_3 stable 2 days ago, uninstalled/installed rb-readline. So my behavior now is different than before (before backspace sometimes worked, now never).

Let's type bash in the dos prompt:

In: 11
Out: 19
Err: 23

Different than above, but IRB broken.

Hope this helps.

Charlie

[luislavena]

This is too crazy and is weird that IRB no longer works for you.

If you don't mind, would you like we do some TeamViewer session? I would really love to experience myself this issue.

Thank you.

[cfis]

Ah - I figured out my IRB issue. Its because back a few months I got readline compiling on windows. So now when I build ruby, it creates readline.so and installs it. However it really doesn't work and causes issues.

So... I deleted readline.so and my backspace key now works most of the time it seems. I assume I'll keep getting the intermittent errors (since I just rebuilt my ruby a few days ago), will post here if I do.

[luislavena]

@cfis: Awesome to hear that! Let's keep an eye and see if we can reproduce this issue with rb-readline ;-)

[rdp]

as a note I still run into this occasionally (probably with console2) with Ruby 1.9.3-p194. Was a fix committed?

[Spakman]

So, have we decided that this is a Console2 bug?

If the workaround patch that @rdp provided doesn't fix all cases, is there something we can do that does?

[luislavena]

So, have we decided that this is a Console2 bug?

Yes, is a Console2 bug and is really hard to reproduce or even document it to report it.