IME support about xi-mac

[kamyuentse]

@pocket7878 You are right, it always report like that XiEditor[8625:49370] (0.0, 0.0, 0.0, 0.0), i'm trying to fix this issue.

[tsekityam]

I will try fixing the implementation of the protocol in this weekend.

[pocket7878]

xi-editor/xi-editor#70

[pocket7878]

Hmm...
Maybe some changes affects to func firstRect(forCharacterRange aRange: NSRange, actualRange: NSRangePointer?) -> NSRect's implementation

[pocket7878]

@jinyuanxie Thank you!

[pocket7878]

It seems cursorPos didn't updated in draw method

[kamyuentse]

@pocket7878 Thank you! I will create a PR later, but there are another problem.
How should we handle the situation that have multi-cursor ?

[pocket7878]

Maybe we should define things like a "active cursor" inside a core and use it for cursorPos

[kamyuentse]

Maybe we should wait backend for supporting "Multiple Cursors" xi-editor/xi-editor#188

[cmyr]

this will definitely be revisited when multi-cursor lands, but I'm not sure how IME would even work if you're inserting text in different contexts simultaneously. If you have any ideas I'm very curious! :)

[tsekityam]

I would like to further improve this behaviour.

The input box should not always follows the cursor. The box should appears at the position of active cursor, then the box's position should be fixed until the input is completed.

That's how the box works in Xcode (and most other editor, if not all),

However, in Xi, the position of the box follows the cursor from time to time,

Current behaviour is a bit annoying because It makes me harder to search for a word in the input box while I am typing.

I will make a PR later tonight.

[kamyuentse]

@tsekityam This behavior may fit XCode by choosing value shown below to update cursorPos orderly

• markRangeStart
• selectedRangeStart
• cix

Or maybe we should define their priority?

edit: a simple patch, it just catch the first appearance of the cursor(s)

Index: XiEditor/EditView.swift
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- XiEditor/EditView.swift	(date 1492533140000)
+++ XiEditor/EditView.swift	(revision )
@@ -192,6 +192,7 @@
             
         }
         // second pass, for actually rendering text.
+        var cursorUpdated:Bool = false
         for lineIx in first..<last {
             // TODO: could block for ~1ms waiting for missing lines to arrive
             guard let line = getLine(lineIx) else { continue }
@@ -205,10 +206,12 @@
             for c in line.cursor {
                 let cix = utf8_offset_to_utf16(s, c)
                 // TODO: How should we handle the situations that have multi-cursor?
-                self.cursorPos = (lineIx, cix)
+                // self.cursorPos = (lineIx, cix)
                 if (markedRange().location != NSNotFound) {
                     let markRangeStart = cix - markedRange().length
                     if (markRangeStart >= 0) {
+                        self.cursorPos = (lineIx, markRangeStart)
+                        cursorUpdated = true
                         attrString.addAttribute(NSUnderlineStyleAttributeName,
                                                 value: NSUnderlineStyle.styleSingle.rawValue,
                                                 range: NSMakeRange(markRangeStart, markedRange().length))
@@ -217,11 +220,18 @@
                 if (selectedRange().location != NSNotFound) {
                     let selectedRangeStart = cix - markedRange().length + selectedRange().location
                     if (selectedRangeStart >= 0) {
+                        if (!cursorUpdated) {
+                            self.cursorPos = (lineIx, selectedRangeStart)
+                            cursorUpdated = true
+                        }
                         attrString.addAttribute(NSUnderlineStyleAttributeName,
                                                 value: NSUnderlineStyle.styleThick.rawValue,
                                                 range: NSMakeRange(selectedRangeStart, selectedRange().length))
                     }
                 }
+                if (!cursorUpdated) {
+                    self.cursorPos = (lineIx, cix)
+                }
             }
 
             // TODO: I don't understand where the 13 comes from (it's what aligns with baseline. We

[tsekityam]

cursorPos is the active cursor position, which should not be related to mark range start or selected range start. We should use mark range start position to calculate the position of the input box.

By the way, I think that the implementation of NSTextInputClient protocol is not correct.

The return values of func markedRange() and func selectedRange() in Xi editor is different from that returns by the TextInputView, the official example that demonstrates how to implement the NSTextInputClient protocol.

Let say we have text apple昌 日日.

The masked range returns by TextInputView is {6, 2}. The location is 6, which is the location of the first 日 in the text; the length is 2, which is the length of the text 日日 .

The selected range returns by TextInputView is {8, 0}. The location is 8, which is the cursor position; the length is 0 because there is nothing selected.

The returned values from xi editor under the same situation are different from Apple's demo app.

Xi editor returns masked range {0, 2}, and selected range {2,0}. I think the ranges returned by Xi editor are started at the beginning of the masked text _日日_, as a result, there is a difference in 6 between those location of ranges provided by TextInputView and Xi editor.

According to the documentation, The returned range measures from the start of the receiver’s text storage., so the ranges returned by Xi editor are not correct.

I suggest that we should review the protocol to ensure that all requirements are met.

If we have the correct implementation, then I think we may be able to calculate the position of the inbox box using the param aRange, which is the masked text range, in firstRect(forCharacterRange:actualRange:), without using the value of cursorPos

[cmyr]

@tsekityam @pocket7878 @jinyuanxie I believe you're right, and the implementation of NSInputClient is not currently correct. Happy to try and help with a fix. If you have any questions, some of us are in channel #xi on irc.mozilla.org.

[tsekityam]

I am almost done, and here is the branch of my patch

I am obtaining the selected range from backend, but there is a problem related to this approach, because of the "insert" requests are async.

Some IME, such as Cangjie, will offer predictive completion.

After the marked text, 日日日, is replaced by the word 晶; the input box will still be there and show the predictive suggestion of the word 晶.

There's how it looks like in EditText:

In TextEdit, the position of predictive completion input box is calculated from the new selection range. The new selected range should be {pos of 晶 + 1, 0}. However, I found that the word 晶 is not inserted immediately after insert request is sent because of the request is async. As a result, the selected range used to calculate the position of the box will never be {pos of 晶 + 1, 0}.

Here is how the completion looks like in my branch

As you can see the completion box is always appeared next to the previous selected range, but not the latest selected range.

If the insertion is not async, I have no way to calculate the position of completion box with the selected range returned from backend. As a result I want to make the insertions request sync.

Besides this issue, the patch is ready for review.

[tsekityam]

I just notice that the predictive completion offered in Xi is different from EditText, even if the input are the same 晶.

I will take a look at this.

[kamyuentse]

selectedRange() and markedRange() seems report the right value.

In my opinion, the previous implement, the _markedRange is relative position of the cursorPos. But the _selectedRange is wrong. This implement, the _markedRange and _selectedRange is relative position of the first line of the viewport, not the whole document.

Although the NSTextInputClient document point out that should be absolute position of the document, but I think use relative position will reduce some calculations.

[tsekityam]

The value of the ranges will be used in other function. e.g. they will be set as the parameter of func firstRect(forCharacterRange range: NSRange, actualRange: NSRangePointer?) -> NSRect, when the app wants to show the IME input box.

If the value of the range are relative position, then firstRect function will receive a relative range, which is not expected. As a result, we have to ignore the parameter being set to the firstRect when we implement it.

That's ok if the IME input box is the one and only one caller of firstRect, selectedRange and markedRange, however, it may not be true.

For example, we may call firstRect when we want to draw a autocompletion box or a popover on a range that is not selected or marked.

Here is one of such popover(!?), that is not drawn on non selected/marked area, in Xcode.

User may expect that we have implemented the protocol correctly when he extends the view. If we don't implement the protocol correctly and he doesn't notice that, then he may be in trouble.

I think it is bad to implement something not following the document by design.

[tsekityam]

By the way, the cause of the strange predictive completion suggestion is that, we didn't implement required protocol correctly.

The IME is using the result of protocol functions, - (NSAttributedString *)attributedSubstringForProposedRange:(NSRange)range actualRange:(NSRangePointer)actualRange; or - (NSAttributedString *)attributedString; to get the word for prediction. But none of them is implemented correctly.

If any of these functions is implemented, then the predictive completion should work.

[kamyuentse]

Here is my thought about IME support:

• Consider the relative or absolute position should be used.
• How to handle multi-selection/cursor?
• Should we use temporary variable to storage the marked text instead of call backend to do insertions. The related behavior can be described as follow:
  1. I selected some text, then type something using InputMethod.
  2. I want to discard the typed text, and go back previous state.
  3. Because currently we call backend to do insert in setMarkedText, we should apply undo more than once to back to previous state.

Also, we should reduce the calculations in draw function which will be call frequently.

[tsekityam]

• How to handle multi-selection/cursor?
  Mark last added selection/cursor as main selection/cursor, and return it from selectedRange().

  If there is marked text to be inserted, then we drop all non-main selection/cursor and only set marked text to main selection/cursor.

  TL;DR: don't support multi selection/cursor in IME input

  ---

  P.S. Atom supports IME input with multiple cursor, however, I don't like their implementation. The IME box position varies while typing, it's hard to locate a word in the box
  

[jansol]

For multiple cursors, the easiest and most intuitive behaviour would probably be showing the IME box at the cursor that was placed last.

[jansol]

Any progress on this? Might be worth opening at least a WIP pull request so others can test and comment on it better.

As for positioning the suggestions, at least for Japanese the segment being edited is further split into subsegments and the suggestion box is aligned with the start of the current subsegment (source: TextEdit, Xcode and Pages). For the multi-cursor scenario I'd stick with my previous suggestion of following the cursor that was placed last, i.e. align with the beginning of the current subsegment starting from the last cursor's position.

[cmyr]

We had this working before the openGL stuff. I believe it isn't working again now, or at least that the IME window is being presented incorrectly.

[jansol]

Currently the IME window is stuck to the bottom left corner so yeah, not presented correctly.