The manga industry often grapples with the intensive labor involved in the creation process, commonly resulting in overworked artists. The meticulous nature of manga drawing typically results in publications featuring black-and-white illustrations. This choice, while traditional, can diminish the reader’s engagement, particularly when an anime adaptation is not yet available. Moreover, while certain art styles are favored, it can be excessively demanding and redundant for artists to reinterpret a series into another style. The implementation of automation through conditional Generative Adversarial Networks (cGANs) for colorization and style transfer via Stable Diffusion model offers a solution. This innovation promises to alleviate the artists’ burden while simultaneously broadening the appeal to diverse audiences, potentially increasing readership and expanding the market. Our initiative aims to foster a greater appreciation of manga as an art form across a wider audience.
While automatic colorization using cGANs has been successfully implemented in previous studies, these implementations often fall short in authentically replicating the actual manga dataset. Typically, textured and grayscale inputs were employed, resulting in idealistic outputs and an ease of achievement that does not accurately reflect authentic manga styles. To address this gap, our report demonstrates the conversion of a colored manga dataset to its sketched version using image processing techniques such as dodging and dilation.
The figure above demonstrates the use of dodging and dilation on grayscaled version of the colored input image to obtain the desired dataset.
Model Used: Pix2Pix condition GAN with residual connections to retain the original input integrity (texts and edges)
The figure below illustrates a marked progression in the quality of colorization with each successive interval of 20 epochs for a total of 80 epochs. Notably, the phenomenon of color spillage observed in the initial stages diminishes over time, and the hues associated with the characters exhibit a dis- cernible deepening in intensity.
Furthermore, the application of our model to unseen data yields commendable results, as evidenced by the vivid and accurate colorization. This outcome is indicative of the model’s robust ability to generalize effectively to novel manga pages.
The experimental phase of our study on style transfer was undertaken utilizing a constrained dataset over a finite duration. Preliminary outcomes, as depicted below, though somewhat irregular in nature, offer a glimpse of the potential advancements achievable in future endeavors. With an expanded dataset and enhanced fine-tuning pro- cesses, these initial results suggest promising directions for further refinement and development in the field of manga style transfer.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent