google / autocjk

AutoCJK is a tool for generating low-resolution predictions of uncommon CJK characters, given full-width images of their components.

AutoCJK

AutoCJK is a tool for generating low-resolution predictions of uncommon CJK characters, given full-width images of their components.

Example:

(Left-to-right: (a) source left-hand component, (b) source right-hand component, (c) expected composition, (d) predicted composition, (e) c/d difference)

How do I generate a new character?

First install the project and its dependencies. See platform-specific instructions on the wiki.

Given a font at ~/Downloads/font.otf, we can render ⿰市來 and write the generated image to /tmp/out.png with:

bazel run //src:main -- \
  --font_path=<path_to_font> \
  --input='⿰市來' \
  --out=/tmp/out.png

Usage guide:

Bundled with this repo is /src/generator.h5, a trained model with pretty good results. This generator was trained on ~270k image pairs, generated by extracting characters from the fonts NotoSansCJKsc-* and NotoSerifCJKsc-*.

The resultant generator is pretty good at generating characters in that style. It might even be pretty good at generating characters in a new style. If you discover it's not, you can (a) generate a new batch of training images in your own font style, and (b) run model.py in order to train a new model. If you save that model as src/generator.h5, you can continue to use bazel run //src:main as expected.

autocjk's Issues

Clearer articulation of region-specific training data, generation, and prediction

Most of the training and testing was performed in cn-zh. But the underlying utilities and fonts certainly support a wider range of regions.

Support characters which don't decompose horizontally.

As a first pass, autocjk trained on and can generate characters with IDSs of the form ⿰XY. But it would be really good to, at a minimum, also support characters with IDSs of the form ⿱XY.

Experiment with other loss models.

Another loss model might make the resultant edges crisper.

Add a model server.

Or use an existing Keras one.

Enable python typed libraries / binaries

This is probably quite simple in Bazel.

Experiment with generating smaller images.

If running generator.py proves slow, it might be good to generate smaller images. Actually, it might be good to make the model configurable.

Write tests for main.py and make_dataset.py.

An integration test which runs continuously on Github Actions would build confidence that this code doesn't break over time.