hell-to-heaven / awesome-sketch-synthesis

:books: A collection of papers about Sketch Synthesis (Generation).

Awesome-Sketch-Synthesis

A collection of papers about Sketch Synthesis (Generation). Mainly focus on stroke-level vector sketch synthesis.

Feel free to create a PR or an issue.

Outlines

• 1. Datasets
• 2. Sketch-Synthesis Approaches
  • 1) Category-to-sketch
  • 2) Photo-to-sketch
  • 3) Text/Attribute-to-sketch
  • 4) 3D shape-to-sketch
  • 5) Sketch(pixelwise)-to-sketch(vector)
  • 6) Art-to-sketch

---

1. Datasets

Here Vector strokes means having svg data. With photos means having the photo-sketch paired data.

Level	Dataset	Source	Vector strokes	With photos	Notes
Characters	Omniglot		✔️	❌	Alphabets characters
	KanjiVG		✔️	❌	Chinese characters
	Kuzushiji		❌	❌	Japanese characters
Instance-level	TU-Berlin	SIGGRAPH 2012	✔️	❌	Multi-category hand sketches
	Sketchy	SIGGRAPH 2016	✔️	✔️	Multi-category photo-sketch paired
	QuickDraw	ICLR 2018	✔️	❌	Multi-category hand sketches
	QMUL-Shoe-Chair-V2	CVPR 2016	✔️	✔️	Only two categories
	Sketch Perceptual Grouping (SPG)	ECCV 2018	✔️	❌	With part-level semantic segmentation information
	FaceX	AAAI 2019	✔️	❌	Labeled facial sketches
Scene-level	Photo-Sketching	WACV 2019	✔️	✔️	ScenePhoto-sketch paired
	SketchyScene	ECCV 2018	❌	✔️	With semantic/instance segmentation information
	CMPlaces	TPAMI 2018	❌	✔️	Cross-modal scene dataset
	Context-Skecth	Expressive 2018	❌	✔️	Context-based scene sketches for co-classification

2. Sketch-Synthesis Approaches

1) Category-to-sketch

Level	Paper	Source	Code/Project Link
Instance-level	A Neural Representation of Sketch Drawings (sketch-rnn)	ICLR 2018	[Code] [Project] [Demo]
	Sketch-pix2seq: a Model to Generate Sketches of Multiple Categories		[Code]
	AI-Sketcher : A Deep Generative Model for Producing High-Quality Sketches	AAAI 2019	[Project]
	Stroke-based sketched symbol reconstruction and segmentation (stroke-rnn)

---

2) Photo-to-sketch

• vector image generation

Level	Paper	Source	Code/Project Link
Facial	Style and abstraction in portrait sketching	TOG 2013
Instance-level	Free-Hand Sketch Synthesis with Deformable Stroke Models	IJCV 2017	[Project] [code]
	Learning to Sketch with Shortcut Cycle Consistency	CVPR 2018	[Code1] [Code2]
	Learning Deep Sketch Abstraction	CVPR 2018

• pixelwise image generation

Level	Paper	Source	Code/Project Link
Instance-level	Deep Factorised Inverse-Sketching	ECCV 2018
	Making better use of edges for sketch generation	JEI 2018
Scene-level	Photo-Sketching: Inferring Contour Drawings from Images	WACV 2019	[Code] [Project]

---

3) Text/Attribute-to-sketch

Level	Paper	Source	Code/Project Link
Scene-level	Sketchforme: Composing Sketched Scenes from Text Descriptions for Interactive Applications	UIST 2019
Facial	Text2Sketch: Learning Face Sketch from Facial Attribute Text	ICIP 2018

---

4) 3D shape-to-sketch

Paper	Source	Code/Project Link
DeepShapeSketch : Generating hand drawing sketches from 3D objects	IJCNN 2019

---

5) Sketch(pixelwise)-to-sketch(vector)

This means translating a pixelwise sketch into a sequential sketch imitating human's drawing order. The appearance of the sequential sketch is exactly the same as the pixelwise one.

Paper	Source	Code/Project Link
Animated Construction of Line Drawings	SIGGRAPH ASIA 2011	[Project] [code] [Demo]

---

6) Art-to-sketch

Here we list sketch synthesis based on other image types, like Manga and line art.

• Hand drawn line art (a.k.a. Sketch Simplification)

Paper	Source	Code/Project Link
Closure-aware Sketch Simplification	SIGGRAPH ASIA 2015	[Project]
Learning to Simplify: Fully Convolutional Networks for Rough Sketch Cleanup	SIGGRAPH 2016	[Code] [Project]
Mastering Sketching: Adversarial Augmentation for Structured Prediction	SIGGRAPH 2018	[Code] [Project]
Real-Time Data-Driven Interactive Rough Sketch Inking	SIGGRAPH 2018	[Code] [Project]
StrokeAggregator: Consolidating Raw Sketches into Artist-Intended Curve Drawings	SIGGRAPH 2018	[Project]
Perceptual-aware Sketch Simplification Based on Integrated VGG Layers	TVCG 2019

• Manga (Comics)

Paper	Source	Code/Project Link
Deep extraction of manga structural lines	SIGGRAPH 2017	[Code]