LINE ARTIST, A Multi-style Sketch to Painting Synthesis Scheme

LINE ARTIST: A Multi-style Sketch to Painting Synthesis Scheme.

Drawing a beautiful painting like what famous painters do is a dream of many people. LineArtist, an interesting system, helps you with drawing a painting only with some semantic sketch.

What you have to do is to draw a sketch, take a photo of it and give it to our pre-trained model, then you can just wait for the painting to come out.

Paper: Jinning Li and Siqi Liu, Mengyao Cao. LINE ARTIST: A Multi-style Sketch to Painting Synthesis Scheme.

[Download Paper]

Video

ABSTRACT

Drawing a beautiful painting is a dream of many people since childhood. In this paper, we propose a novel scheme LINE ARTIST to synthesize artistic style paintings with freehand sketch images, leveraging the power of deep learning and advanced algorithm.

Our scheme includes three models. The Sketch Image Extraction (SIE) model is applied to generate the train- ing data. It includes smoothing reality images and pen- cil sketch extraction. The Detailed Image Synthesis (DIS) model trains a conditional adversarial net to generate re- ality detailed information. The Adaptively Weighted Artis- tic Style Transfer (AWAST) model is capable to combine multiple style image with content via VGG19 network and PageRank algorithm. The appealing stylized images are then generated by optimization iterations.

Experiments are operated on the Kaggle Cats dataset and The Oxford Buildings Dataset. Our synthesis results are artistic, beautiful and steadier. Real sketch tests prove that our scheme performs very well on reality environments. With different artist styles, our scheme can generate paintings of various styles. With our scheme, everyone can draw a beautiful picture only by drawing a sketch and then feeding it into our system.

contentgoldbridgeourgoldbridge

Source Code

The code is published in github: https://github.com/jinningli/LineArtist

Requirements

Python 3
Pillow (4.2.1)
numpy (1.13.3)
scipy (0.19.0)
opencv-python (3.3.0.10)
tensorflow (1.3.0)
Matlab

Generate Dataset Using Sketch Image Extraction(SIE) Model:

Put your image in the folder ./SIE/SourceImage, then run these command:

$ cd ./SIE
$ python3 ./preprocess.py

Follow the instructions, dataset will be built in the folder ./SIE/Datasets

Synthesize reality image using Detailed Image Synthesis(DIS) Model:

Please put your dataset inside the folder ./DIS/Datasets #### Train

$ cd ./DIS
$ python3 train.py --dataroot ./Datasets/[NAME] --model pix2pix --which_direction AtoB --name [NAME] --gpu_ids 0

Test

$ cd ./DIS
$ python3 test.py --dataroot ./Datasets/[NAME] --model pix2pix --which_direction AtoB --name [NAME] --gpu_ids 0

All the checkpoints will be saved in ./DIS/checkpoints. The result will be saved in ./DIS/Results.

Stylize using Adaptive Weighted Artist Style Transfer(AWAST) Model:

$ cd ./AWAST
$ python3 AWAST.py --content [Path] --folder_styles [Path] --output [Path]

Reference

junyanz/pix2pix

anishathalye/neural-style