Code and data for paper "Deep Photo Style Transfer"
This code is based on torch. It has been tested on Ubuntu 14.04 LTS.
Dependencies:
CUDA backend:
Download VGG-19:
sh models/download_models.sh
Compile cuda_utils.cu
(Adjust PREFIX
and NVCC_PREFIX
in makefile
for your machine):
make clean && make
To generate all results (in examples/
) using the provided scripts, simply run
run('gen_laplacian/gen_laplacian.m')
in Matlab or Octave and then
python gen_all.py
in Python. The final output will be in examples/final_results/
.
- Given input and style images with semantic segmentation masks, put them in
examples/
respectively. They will have the following filename form:examples/input/in<id>.png
,examples/style/tar<id>.png
andexamples/segmentation/in<id>.png
,examples/segmentation/tar<id>.png
; - Compute the matting Laplacian matrix using
gen_laplacian/gen_laplacian.m
in Matlab. The output matrix will have the following filename form:gen_laplacian/Input_Laplacian_3x3_1e-7_CSR<id>.mat
; - Run the following script to generate segmented intermediate result:
th neuralstyle_seg.lua -content_image <input> -style_image <style> -content_seg <inputMask> -style_seg <styleMask> -index <id> -serial <intermediate_folder>
- Run the following script to generate final result:
th deepmatting_seg.lua -content_image <input> -style_image <style> -content_seg <inputMask> -style_seg <styleMask> -index <id> -init_image <intermediate_folder/out<id>_t_1000.png> -serial <final_folder> -f_radius 15 -f_edge 0.01
Note: In the main paper we generate all comparison results using automatic scene segmenation algorithm modified from DilatedNet. Manual segmentation enables more diverse tasks hence we provide the masks in examples/segmentation/
.
Here are some results from our algorithm (from left to right are input, style and our output):
- Our torch implementation is based on Justin Johnson's code;
- We use Anat Levin's Matlab code to compute the matting Laplacian matrix.
Feel free to contact me if there is any question (Fujun Luan [email protected]).