Neural networks for colorizing black & white photos automatically
The code in this program is a reproduction of the following article:
https://blog.floydhub.com/colorizing-b-w-photos-with-neural-networks/
The goal is to colorize black & white photos automatically.
First, I use a specific variety of cat -- Siamese kitten, to train the beta-version neural network proposed in the above article. I use 200 images to train this 16-layers neural network. The results seems acceptable and reasonable.
Second, I use different kinds of cat & dog images to train the full-version neural network proposed in the above article, which fuses the classification networks output with the encoder results then input decoder together. The classifier is "the inception resnet V2". This enables the network to match an object representation with a colouring scheme, so image features can be better understood. So I can input training images containing different varieties of cats, dogs and other objectives, let the network classifies, and I can still get a acceptable results. And this time, considering my computer's limited memory space and GPU performance, I only choose 60 images for training.
colornet_beta.py: train and test for model without classification network.
colornet_final.py: train and test for model with classification network.
preprocess.py: input images pre-procession.
final: part of output pictures
structure: the structure introduction of two version networks
Highlight:
- using 2 strides conv filter to replace max-pooling, in case of distoration
- activation function: relu, except for output layer: tanh
- In the
ImageDataGenerator, we adjust the setting for our image generator. This way, one input image will never be the same. - colorspace: Lab
- Loss: mean square error (1000 epoch final average loss: 8e-4)
- model size: 29.7M
Highlight:
- split whole network into four parts: encoder, classifier, fusion layer and decoder
- add classification network output (1*1*1000) to encoder output (32*32*256), and together as the input of decoder
- loss: mean square error (1000 epoch final average loss: about 0.0030)
- model size: 74.9M
At first, I train this network with 500 images of different variety of cats&dogs for 100 epoch.
The test outputs truned out bad, the whole picture is in brown.
Input -> Output -> Ground Truth:
When this network just learn the pictures of siamese kitten, test results become better:
It shows particular features like blue eyes, brown neck, green or blue background.
But if this network is used on generalized cats and dogs colorization, the results are poor even sometimes ridiculous:
After adding extra classifier to beta version network, this mixed-network should enable to identify more objects hence becoming more generic.
I train this network with only 60 images of cats&dogs for 1000 epoch.
The test set is comprised of 100 images, most of them perform badly (just brown covers a large area on animals), but I still can choose some good results from them.
It seems like it is goot at colorizing lawn and grass. But unfortunately it sometimes mistakes animal fur for grass:
Restricted by my laptop GPU and memory space, I don't train final version network on a larger training set, at least up to thousands of pictures. As a result, the mean square error is not low enough.
- Kaggle Dog vs. Cat database（select 60 images for train，100 images for test）
- google: Siamese kitten, bulk download by Chrome extensions: Fatkun (200 images for train, 30 images for test)
Zoom the training images to 256*256 pixel size, using black pixels as padding.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent