when open sqlite3.c, it is crash about xi-mac

[fanfeilong]

@cmyr @vlovich The files I am trying to open are in my local project. It seems that any file in this folder will crash. I have tar the test source files and crash report file with tar cvzf:
crash_report.tar.gz

[cmyr]

Strange. Some more information would be useful:

• Are there any relevant crash reports in ~/Library/Logs/DiagnosticReports?
• Did you build from the command line with xcodebuild, or via Xcode.app?
• Other files are working fine?

[vlovich]

Also can you attach the file or provide a URL? I grabbed https://raw.githubusercontent.com/xamarin/sqlite/master/dist/sqlite3.c & it opened without issue for me.

[vlovich]

func utf8_offset_to_utf16(_ s: String, _ ix: Int) -> Int {
return s.utf8.index(s.utf8.startIndex, offsetBy: ix).encodedOffset
}

Precondition failed: file /BuildRoot/Library/Caches/com.apple.xbs/Sources/swiftlang/swiftlang-900.0.74.1/src/swift/stdlib/public/core/StringUTF8.swift, line 185
2018-01-31 20:23:56.660028-0800 XiEditor[54308:17000757] Precondition failed: file /BuildRoot/Library/Caches/com.apple.xbs/Sources/swiftlang/swiftlang-900.0.74.1/src/swift/stdlib/public/core/StringUTF8.swift, line 185

po s.count
80

ix 83

Looks like an out of bounds issue of some kind I think.

[vlovich]

#3 0x00000001000315ad in utf8_offset_to_utf16(::) at /Users/vlovich/src/xi-mac/XiEditor/StyleMap.swift:105
#4 0x0000000100030fcc in static StyleSpan.styles(fromRaw:text:) at /Users/vlovich/src/xi-mac/XiEditor/StyleMap.swift:91
#5 0x000000010003f4e6 in Line.init(fromJson:) at /Users/vlovich/src/xi-mac/XiEditor/LineCache.swift:44
#6 0x0000000100044f28 in LineCacheState.applyUpdate(update:) at /Users/vlovich/src/xi-mac/XiEditor/LineCache.swift:148
#7 0x000000010004be5b in LineCacheLocked.applyUpdate(update:) at /Users/vlovich/src/xi-mac/XiEditor/LineCache.swift:326

[vlovich]

StyleSpan.styles:
(lldb) po text.count
80

(lldb) po start
3

(lldb) po end
83

(lldb) po style
2

[cmyr]

okay, this is because we aren't correctly handling the BOM. Adding the BOM to an otherwise okay file results in the same crash.

This highlights some general laxness we have around handling character encoding.

[cmyr]

fixed in xi-editor/xi-editor#513