2022
NeurIPS
NeurIPS 2022
CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image Encoders
Abstract
CLIPDraw is an algorithm that synthesizes novel drawings from natural language input. It does not require any additional training; rather, a pre-trained CLIP language-image encoder is used as a metric for maximizing similarity between the given description and a generated drawing. Crucially, CLIPDraw operates over vector strokes rather than pixel images, which biases drawings towards simpler human-recognizable shapes. Results compare CLIPDraw with other synthesis-through-optimization methods, as well as highlight various interesting behaviors of CLIPDraw.
🌉
Interdisciplinary Bridge
- Artificial Intelligence and Computer Science and Computer Vision and Mathematics & Optimization and Natural Language Processing
Authors
Topics
Machine Learning > Optimization & Theory > Optimization
Machine Learning > Application Areas > Efficient Computing
Deep Learning > Models > Generative Models
Computer Vision > Generation > Image Generation
Natural Language Processing > Generation > Text Generation
Mathematics & Optimization > Optimization > Continuous Optimization
Computer Science > Applications > Information Retrieval
Computer Science > Applications > Computer Graphics