janlelis / uniscribe

Know your Unicode ✀

uniscribe | Describe the Unicode

Describes Unicode characters with their name and shows compositions. UNICODE 15.1*

• Helps you understand how glyphs and codepoints are structured within the data
• Gives you the names of glyphs and codepoints, which can be used for further research
• Highlights invalid/special/blank codepoints

Uses a similar color coding like its lower-level companion tool unibits.

Setup

Make sure you have Ruby installed and installing gems works properly. Then do:

$ gem install uniscribe

Usage

Pass the string to debug to uniscribe:

From CLI

$ uniscribe "test strı̈ng"

From Ruby

require "uniscribe/kernel_method"
uniscribe "test strı̈ng"

Output

0074 ├─ t		├─ LATIN SMALL LETTER T
0065 ├─ e		├─ LATIN SMALL LETTER E
0073 ├─ s		├─ LATIN SMALL LETTER S
0074 ├─ t		├─ LATIN SMALL LETTER T
0020 ├─ ] [		├─ SPACE
0073 ├─ s		├─ LATIN SMALL LETTER S
0074 ├─ t		├─ LATIN SMALL LETTER T
0072 ├─ r		├─ LATIN SMALL LETTER R
---- ├┬ ı̈		├┬ Composition
0131 │├─ ı		│├─ LATIN SMALL LETTER DOTLESS I
0308 │└─ ◌̈		│└─ COMBINING DIAERESIS
006E ├─ n		├─ LATIN SMALL LETTER N
0067 ├─ g		├─ LATIN SMALL LETTER G

Examples

Tamil

>> uniscribe "நகரத்தில்"

Thai

>> uniscribe "ม้าลายหกตัว"

Ideographic Variations

>> uniscribe "辻󠄀㚑󠄁"

(the variation is not visible in the screenshot, because my system does not render it correctly)

Emoji Sequences

>> uniscribe "3️⃣🤸‍♀"

Lots of Combining Marks

>> uniscribe "̶̧̨̱̹̭̯ͧ̾ͬC̷̙̲̝͖ͭ̏ͥͮ͟Oͮ͏̮̪̝͍"

Random Sequences of some Special Unicode Codepoints

>> uniscribe "\0A\u{E01D7}\x7F\r\n\u{D0000}\u{81}\u{FFF9}B\u{FFFB}🏴\u{E0061}\u{E007F}\u{10FFFF}"

Some Blanks

>> uniscribe "­ᅠ 𝅸"

*Notes

Although the gem is generally up to date with Unicode 15.0, the proper detection of compositions / graphemes / combined characters depends on your Ruby version:

You can run uniscribe -v to check for the Unicode level of your uniscribe version.

Also see

• CLI: unibits - visualizes Unicode encodings
• CLI: unicopy - copy codepoints to clipboard
• Website: character.construction - lists notable codepoints
• Ruby Library: symbolify - used for safely printing individual codepoints
• Ruby Library: characteristics - used for detecting blanks and similar
• Unicode® Standard Annex #29: Unicode Text Segmentation
• Talk: Ten Unicode Characters You Should Know About as a Programmer

Copyright (C) 2017-2023 Jan Lelis https://janlelis.com. Released under the MIT license.

uniscribe's Issues

How are compositions detected?

Hey!

This is an amazing tool! I'm wondering how I could detect if a character is composited in my own code and treat it as a single character, not as multiple ones. I found a few rules, but there keep being new ones. Is there a general-purpose way of finding out?