ibicha / emojitexture

A Unity plugin to render Emojis ☺ ❤ 🍆 🍑 to a texture

EmojiTexture

⚠️ Unity now supports Emojis using TextMesh Pro. Please use that package instead, since this project hasn't been updated for a while.

A Unity plugin to render Emojis ☺ ❤ 🍆 🍑 to a texture. Currently for iOS and Android only.

Please note that the editor is not supported. It will only render on device (should work on simulator as well)

Preview

Usage

As simple as:

material.mainTexture = new EmojiTexture("❤");

You can also do these things:

//Create an EmojiTexture with a specific size (best if power of 2)
var emojiTexture = new EmojiTexture(128);

//Change an existing EmojiTexture
emojiTexture.Text = "❤"; 

//Get the texture as an array of bytes, in case you want to do something with it
var bytes = emojiTexture.ByteBuffer;

//Know the code of the emoji? Set it directly as an integer!
//E.g. https://emojipedia.org/smiling-face-with-smiling-eyes/
emojiTexture.Unicode = 0x1F60A; //😊 Smiling Face With Smiling Eyes

Github emoji API (Experimental)

When not running Android or iOS, EmojiTexture can use Github emoji API. This require network connection. While the list of emojis is cached, individual images are not.

Setting emojis from Github needs to run in a Coroutine, because it is an async operation. It is possible to set the emoji text (same as above)

yield return emojiTexture.SetGithubEmoji("❤");

Or using keywords

yield return emojiTexture.SetGithubEmoji(":heart:");

Please check out example scene for usage (it includes a native emoji, a github emoji and TextMesh Pro examples, that shuffle emojis from script or read user imput with touch screen).

TextMesh Pro support (Experimental)

TextMesh Pro already supports emojis as sprites, but they need to be prepared beforehand, which makes it troublesome in terms of build size (and also a lot of manual work), if you want to support as many emojis as possible. This is where EmojiTexture comes in. It generates these sprites on the fly as they are needed.

Setup

- Import TextMesh Pro from the Asset Store Use the new Package Manager to install TextMesh Pro (this should happen automatically since it is a project dependency)

• In the Player Settings, add TMPRO_EMOJIS to the Scripting Define Symbols

• Add the component TMP_EmojiSupport on the same game object as your TextMeshPro or TextMeshProUGUI component. That's about it. You should have emoji support out of the box.
• On the TMP_EmojiSupport component, if Github fallback is checked, the emojis will be downloaded from github if you are not running on Android or iOS (and if you have network)

Optimizations

Few pointers to consider:

• Emojis are stored in sprite sheets of the size 4 by 4 (16 emojis) with an emoji texture of 128x128 pixels (which makes a 512x512 per sprite sheet.) These are constants defined in TMProEmojiAsset.EMOJI_SIZE and TMProEmojiAsset.SHEET_TILES. Optimize these values according to your use case. BE AWARE currently, github returns enoji images as 128x128, and they share the same sprite sheet. Changing these constants will probably break emojis from github. (A resizing method needs to be implemented for that purpose, but that would be at the cost of performance)

• When a sprite sheet is full, a new one is created.

• Currently cleaning up unused emojis is not yet supported. This will be added in the future.

Known issues.

• While the native emoji renderer (Android/iOS) tries to estimate the correct rendering, Github emojis and TMP both have trouble with complex emojis (such as flags, emojis with skintones, etc). These will be addressed in the future, as there is room for improvement.
• Some rendering issues are due to how TMP works, because it scans for unicode characters, and we get to feed it sprites. This needs to be fixed on TMP's side.

emojitexture's Issues

Image caching

Implement a simple cache mechanism to save downloaded images, and clear cache when needed (maybe an expiry as well).

Fix code points sequence rendering (emoji modifiers) (faulty Regex)

Combinations are currently not working.
E.g. an emoji of man + skin tone is not detected as a single emoji, but as 2.

Using text change events

"Hooking" to textmesh pro is too awkward. Should be a behaviour that is added with a TMP text, leveraging the TMPro_EventManager.TEXT_CHANGED_EVENT event.

GitHub Like Emoji

Using GitHub like emoji codes for instantiating EmojiTexture class, for example instead of doing:

material.mainTexture = new EmojiTexture("❤");

We can do this:

material.mainTexture = new EmojiTexture(":heart:");

Thanks.

[Research] AppleColorEmoji TOS

Stumbled on https://www.reddit.com/r/iOSProgramming/comments/6zpxms/use_of_apple_emoji_against_tos_in_ios_app_but/

Need to research TOS before moving on, and put any needed disclaimers in the README.

IUnityRenderingExtensions.h file not found

IUnityRenderingExtensions.h file not found error when building with Unity3D 2018.2.6f1 and Xcode 9.4.1

Emoji modifiers in TextMesh Pro

TextMesh Pro doesn't not support Emoji modifiers yet. Need work around.

IOS14 stop working

Someone investigate this problem?
When on keyboard switch to smile get errors:

2020-11-24 12:05:27.446448+0200 The behavior of the UICollectionViewFlowLayout is not defined because:

2020-11-24 12:05:27.446534+0200 the item height must be less than the height of the UICollectionView minus the section insets top and bottom values, minus the content insets top and bottom values.

2020-11-24 12:05:27.446773+0200 The relevant UICollectionViewFlowLayout instance is <_InvertibleFlowLayout: 0x127fcfde0>, and it is attached to <UICollectionView: 0x12e335a00; frame = (0 0; 896 414); clipsToBounds = YES; autoresize = W+H; gestureRecognizers = <NSArray: 0x28059da70>; layer = <CALayer: 0x2813d7120>; contentOffset: {0, 0}; contentSize: {0, 0}; adjustedContentInset: {0, 0, 0, 0}; layout: <_InvertibleFlowLayout: 0x127fcfde0>; dataSource: <__UIDiffableDataSource 0x282800ff0: sectionCounts=[_UIDataSourceSnapshotter - 0x2813cdea0:(0:169)]; sections=[0x2813ce3a0]; identifiers=[0x2813cdf80]>>.
2020-11-24 12:05:27.446815+0200 Make a symbolic breakpoint at UICollectionViewFlowLayoutBreakForInvalidSizes to catch this in the debugger.

2020-11-24 12:05:55.502103+0200 CoreText note: Client requested name ".PhoneFallback", it will get TimesNewRomanPSMT rather than the intended font. All system UI font access should be through proper APIs such as CTFontCreateUIFontForLanguage() or +[UIFont systemFontOfSize:].

2020-11-24 12:05:55.502209+0200 CoreText note: Set a breakpoint on CTFontLogSystemFontNameRequest to debug.

Use DirectWrite on windows

Native rendering on windows using DirectWrite

Use IssuePluginCustomTextureUpdate

Could use IssuePluginCustomTextureUpdate for rendering on mobile since it supports GLES and Metal

Custom tags

Something like <emoji="heart"> or maybe would be a nice addition.

Render text without a UILabel

I know it is not efficient, and probably unnecessary, to use a UILabel for rendering the text, but it was a fast mean to get an accurate rendering, without going through calculating the correct size of the glyphs.

WebGL support

test.html :

<!DOCTYPE html>
<html>
<body>
<script>
	var DEBUG = true;
	function Render(emojiText, width, height, /*ignored*/sanitize) {
		var canvas = document.getElementById("emojiCanvas") || document.createElement("canvas");

		canvas.id = "emojiCanvas"

		if(DEBUG)
			document.body.appendChild(canvas);

		canvas.width = width;
		canvas.height = height;

		var ctx = canvas.getContext("2d");
		ctx.textAlign = "center";
		ctx.textBaseline = "middle"
		ctx.font = "256px sans-serif";

		var size = canvas.width;
		var scaleFactor = size / ctx.measureText(emojiText).width;

		ctx.translate(size/2, size/2);
		ctx.scale(scaleFactor, scaleFactor);
		ctx.translate(-size/2, -size/2);

		ctx.fillText(emojiText, size/2, size/2);

		return ctx.getImageData(0, 0, width, height).data;
	}
	
	function DoRender() {
		var emojiText = "👨‍🌾";
		var size = 64;
		Render(emojiText, size, size);
	}

	DoRender()
</script>
</body>
</html>

Missing Emojis - Tested on Android

Really great project but I notice some emojis are missing when I test on Android (not yet tested iOS) with the Github Fallback disabled. 😈 (U+1F608) 🦓 (U+1F993) for example display as squares. Is this an issue my end or a known bug?

Also, is there any update on complex emojis such as 💂 U+1F482. They also still display incorrectly.

I would really like to deploy this to my app, could do wonders for the user experience. Many thanks for all the progress so far.

Emojis on Android are flipped with a wrong offset

Hi,
Your plugin is really great! But sadly on Android marshmalow 6.0 (tested on 2 phones).
I got some weird behaviors:

Can you tell me if you have the same issues and if you have an idea on how to fix this?
The plugin work well on Andoid 9.0 :)
Otherwise on iOS (10.3.3) the TextMeshPro part doesn't work :'(
I see only the unsupported text character... If you have an idea on this too?

Thanks!

Totally broken on iOS

Some code got forgotten in the last commit for the iOS plugin. FIX ASAP

NoSuchMethodError on android 12 and above

Error:

java.lang.NoSuchMethodError · no non-static method "Ljava/nio/DirectByteBuffer;.(JI)V"

Stacktrace:

Native Method com.unity3d.player.UnityPlayer.nativeRender()
Unknown Source com.unity3d.player.UnityPlayer.access$300()
Unknown Source:95 com.unity3d.player.UnityPlayer$e$1.handleMessage()
Handler.java:102 ndroid.os.Handler.dispatchMessage()
Looper.java:226 ndroid.os.Looper.loopOnce()
Looper.java:313 ndroid.os.Looper.loop()
Unknown Source:20 com.unity3d.player.UnityPlayer$e.run()
unknown UnityEngine.AndroidJNISafe.CheckException()
System.IntPtr obj, System.String name, System.String sig UnityEngine.AndroidJNISafe.GetMethodID()
System.IntPtr jclass, System.String signature UnityEngine._AndroidJNIHelper.GetConstructorID()
System.String className, System.Object[] args UnityEngine.AndroidJavaObject._AndroidJavaObject()
System.String text, System.Int32 width, System.Int32 height iBicha.EmojiTexture..ctor()
TMPro.TMP_Text tmp_Text iBicha.TMPro.TMProEmojiAsset.HookTMP()

Crash in iOS plugin when entering Multi-Character Emoji

Hey.

I found a crash in the iOS Plugin. Currently unfortunately I cannot give you debug data, but i can give you reproduction steps.

1. Create a TMP Emoji Input field.
2. Build scene to iOS Device.
3. Start Entering Hand/Gesture Emojis 👍✌🤞🤞🙌👍🤦‍♀️🙌👏👏👏
4. When you get to 8 Emojis, on the 9th entered one, UTF32 sent to plugin will overflow and cause a fatal crash.

I got this fixed by changing the way the emoji is retrieved ( instead of using the character-based search, i changed it to GetFirstEmoji() ). I cannot post the source, unfortunately, and also it's kindof a bodgejob so it's probably for the best.

Cheers!

CoreText on iOS/macOs

Cannot Convert the int to Uint

Hi there I Imported your Project in my project the Issue I Face is it is showing the Error Cannot Change the int into UInt
There is the Images Below for the Reference Any type of help.

Android 64 Bits Support?

Hello, I just realized when compiling for Android 64 bits, it crashes.

Unicode property

Should be possible to set the text with an int representing the unicode value of the emoji.

Emoji doesn't render on 512x512 textures

512x512 textures and larger do not display emojis. They do display regular characters though.

Android Api level 31 fix?

It works fine in api level 30 but in 31 it crashes, and Google requires Api level 31 from November. Any help?

TextMesh Pro support

Automatic generation of sprite sheets to be used by TextMesh Pro.

Auto detect TextMeshPro through CCU

https://github.com/Unity-Technologies/ConditionalCompilationUtility

[EmojiTexture v2] SkiaSharp/HarfBuzzSharp integration

Motives:
When I first created this plugin, I wanted to keep away from all the complexities related to font/ glyphs, rendering, and so on, and leave that for the underlying OS to do the heavy lifting.
This resulted in a flaky implementation that had platform specific bugs (and even per os version).

What's next:
The superior solution was to implement a library that is capable of loading emojis straight from color font files and render them directly. The implementation was delayed because of many posts/news about TextMesh Pro was in progress with such features. But since the wait got too long, and the lack of a good solution is becoming frustrating to some people, I resumed working on this.

The how:
The idea is to leverage a combination of great libraries (FreeType/Harfbuzz/Cairo SkiaSharp/HarfBuzzSharp) that do this pretty well, and use that for EmojiTexture. This will allow better flexibility, quality and performance in the future.

Currently in the TODO list:

• Testing a native solution around these libs (Emoji+modifiers, different resolutions, etc)
• Rewriting the library from the ground up
• Investigating any platform-specific issue (e.g. permission to access font files on mobile)
• Checking platform support for used libraries

Sidenote: TextMesh Pro will have colored glyphs with Unity 2019.2, and compatible with 2019.1, according to the roadmap (https://youtu.be/qwGjneorKgs?t=2742). It sounds like this feature is not going to be supported on earlier versions of Unity.

Refactor to EmojiProvider

Refactor current code to implement interface EmojiProvider (native, github, other) and possibility to set fallback list