LINE ARTIST, A Multi-style Sketch to Painting Synthesis Scheme
LINE ARTIST: A Multi-style Sketch to Painting Synthesis Scheme.
Drawing a beautiful painting like what famous painters do is a dream of many people. LineArtist, an interesting system, helps you with drawing a painting only with some semantic sketch.
What you have to do is to draw a sketch, take a photo of it and give it to our pre-trained model, then you can just wait for the painting to come out.
Paper: Jinning Li and Siqi Liu, Mengyao Cao. LINE ARTIST: A Multi-style Sketch to Painting Synthesis Scheme.
Video
ABSTRACT
Drawing a beautiful painting is a dream of many people since childhood. In this paper, we propose a novel scheme LINE ARTIST to synthesize artistic style paintings with freehand sketch images, leveraging the power of deep learning and advanced algorithm.
Our scheme includes three models. The Sketch Image Extraction (SIE) model is applied to generate the train- ing data. It includes smoothing reality images and pen- cil sketch extraction. The Detailed Image Synthesis (DIS) model trains a conditional adversarial net to generate re- ality detailed information. The Adaptively Weighted Artis- tic Style Transfer (AWAST) model is capable to combine multiple style image with content via VGG19 network and PageRank algorithm. The appealing stylized images are then generated by optimization iterations.
Experiments are operated on the Kaggle Cats dataset and The Oxford Buildings Dataset. Our synthesis results are artistic, beautiful and steadier. Real sketch tests prove that our scheme performs very well on reality environments. With different artist styles, our scheme can generate paintings of various styles. With our scheme, everyone can draw a beautiful picture only by drawing a sketch and then feeding it into our system.
Source Code
The code is published in github: https://github.com/jinningli/LineArtist
Requirements
Python 3
Pillow (4.2.1)
numpy (1.13.3)
scipy (0.19.0)
opencv-python (3.3.0.10)
tensorflow (1.3.0)
MatlabGenerate Dataset Using Sketch Image Extraction(SIE) Model:
Put your image in the folder ./SIE/SourceImage, then run these command:
$ cd ./SIE
$ python3 ./preprocess.pyFollow the instructions, dataset will be built in the folder ./SIE/Datasets
Synthesize reality image using Detailed Image Synthesis(DIS) Model:
Please put your dataset inside the folder ./DIS/Datasets
Train
$ cd ./DIS
$ python3 train.py --dataroot ./Datasets/[NAME] --model pix2pix --which_direction AtoB --name [NAME] --gpu_ids 0Test
$ cd ./DIS
$ python3 test.py --dataroot ./Datasets/[NAME] --model pix2pix --which_direction AtoB --name [NAME] --gpu_ids 0All the checkpoints will be saved in ./DIS/checkpoints. The result will be saved in ./DIS/Results.
Stylize using Adaptive Weighted Artist Style Transfer(AWAST) Model:
$ cd ./AWAST
$ python3 AWAST.py --content [Path] --folder_styles [Path] --output [Path]