SickZil-Machine-DirectML

^{English | 한국어 | 汉语}
Forked from KUR-creative/SickZil-Machine.

Notice

SZMC 0.1.1 - Fixed link
Tutorials and Tips

Currently, I am still working on the SickZil-Machine. It is just laying the groundwork invisible to users. I'm building the system I need for more effective training data collection, more efficient deep learning model training, faster experimentation and analysis, model serving and deployment automation, etc.
I will go to the distance. I appreciate if you could wait a little longer. Thank you.

^{(source: manga109, © Kanno Hiroshi, © Okuda Momoko, © Kato Masaki)}

SickZil-Machine automates texts removal during manga/comics translation(Scanlation) process.

All of the above images were edited automatically without human intervention.
^{(source: manga109, © Shimazaki Yuzuru, © Akamatsu Ken, © Kobayashi Yuki)}

How it works??

Model

SickZil-Machine finds out the texts in manga/comics and erases it naturally to match the background.
Both processes are completely automatic, without any additional human intervention.
Of course, if you want, you can also specify text area you want to erase.

_{By the way, SickZil is korean word 식질, slang of 식자(작업).
식자 means editing manga/comics according to the translation(from translator).}

We applied U-net for SegNet and Deepfill v2 for ComplNet.

Data set

SickZil-Machine consists of two deep learning models, SegNet and ComplNet.

To learn SegNet, we need original manga images and
text component masks that cover all text area corresponding to the original images.

To learn ComplNet, we need manga images with text removed (ie output).
(I'm researching how an images with a small amount of text affects performance.
manga images with no text at all are the ideal data.)

Version 0.1.1 was trained using 285 image-mask pairs and 31,497 manga images.
(11,464 of 31,497 manga images are images with text.)

If you'd like to contribute a dataset to SickZil-Machine, please send your data to email .
The dataset will only be used for research purposes.

Release (not applicable to DirectML, see "Running code (for developers)")

We released 0.1.1 pre-release version!
You can download SZMC here.
Tutorials and Tips here.

SickZil-Machine is not a perfect program. We need your help.
If you find a bug or have a suggestion, please open a Github issue or send us an email.

Run the code(for developers)

The TensorFlow with DirectML package on native Windows works starting with Windows 10, version 1709 (Build 16299 or higher). You can check your build version number by running winver via the Run command (Windows logo key + R). (tensorflow-directml 1.15.0 requirements)

git clone https://github.com/Aloereed/SickZil-Machine-DirectML.git; cd SickZil-Machine-DirectML
Download one of release zip files from here.
Unzip the release file and copy SickZil-Machine-0.1.1-pre0-win64-cpu-eng/resource/cnet and SickZil-Machine-0.1.1-pre0-win64-cpu-eng/resource/snet directories to SickZil-Machine-DirectML/resource.
cd src; pip install -r requirements.txt
python main.py

Future works

Increase text segmentation performance
Open manga text segmentation mask dataset
Automate typesetting(calligraphy style learning)

aloereed / sickzil-machine-directml Goto Github PK