Cursor never moves forward when mixing autocomplete and emoji about libtsm

[Aetf]

Hmm, another weird bug with emoji.

I don't have zsh installed, but I'm using fish, which has autocomplete builtin. Anyway, I tested in kmscon, gtktsm, and also in non-libtsm-based konsole, just to see how does it behave.

Despite some font and other cursor placement issues, I can't reproduce the exact issue you described.

Probably due to the way zsh implementing autocomplete is different from fish?

[refi64]

Just tried, and I can't reproduce it in fish either.

Xonsh is another shell where this problem comes up.

[Aetf]

I currently have few spare time to investigate this. Pull requests are welcomed.

[refi64]

Hmm, looks like the problem is basically with mk_wcwidth: the library is old, and the Unicode standards it's based on haven't been updated in years. Therefore, is basically thinks that most characters added after 2007 (including emoji) are only 1 column wide, which screws up all the calculations.

Out of curiosity: is there a reason libtsm doesn't just use POSIX's wcwidth? For all practical purposes, on Unix platforms, it's usually UTF-32/UCS4 anyway. There's also a wcwidth implementation that Termux uses...

[Aetf]

It's on the original author's to-do list for a while...

Anyway, I tried the wcwidth implementation from Termux, which seems to be the most up to date one in C (usually POSIX's wcwidth in libc is several Unicode standards behind).
Now the emoji is displayed full width, but breaking line width calculation in fish. But xonsh is working fine now...

I pushed the code to wcwidth branch, if you want to play with it.

[refi64]

@Aetf I believe this is this Fish bug, as I can reproduce it in Sakura and URxvt, too.

One of the comments seems to suggest that this happens when the terminal emulator's wcwidth disagrees with Fish's. In this case, Fish thinks the emoji is width 1, but now libtsm is (correctly) seeing it as width 2, which breaks everything.

Maybe there could be a runtime setting for this? Some kind of global function (tsm_set_internal_wcwidth?) that sets whether or not to globally use the internal wcwidth or the system one. That way, when on shells like Fish that glitch out, you could override the setting.

[Aetf]

Yeah, I saw that bug a while ago when I was messing around with emojis in Konsole.

Having a switch for wcwidth isn't going to help here, as fish also uses its internal copy of wcwidth, unless otherwise compiled.

Another way is to force fish emoji width to 2 using set fish_emoji_width 2. I tried that with gtktsm (using updated wcwidth), and it works, at least the cursor position is correct.

Does the wcwidth branch fixes the original problem you posted? If so I'm going to merge that.

[refi64]

Yup, it fixes the original issue. I guess Xonsh works now since Python probably has its own wcwidth.

[Aetf]

Xonsh works with new wcwidth. I've tried that. Fix merged in #8