Scones

Iteratively refining and critiquing sketches are crucial steps to developing\neffective designs. We introduce Scones, a mixed-initiative,\nmachine-learning-driven system that enables users to iteratively author\nsketches from text instructions. Scones is a novel deep-learning-based system\nthat iteratively generates scenes of sketched objects composed with semantic\nspecifications from natural language. Scones exceeds state-of-the-art\nperformance on a text-based scene modification task, and introduces a\nmask-conditioned sketching model that can generate sketches with poses\nspecified by high-level scene information. In an exploratory user evaluation of\nScones, participants reported enjoying an iterative drawing task with Scones,\nand suggested additional features for further applications. We believe Scones\nis an early step towards automated, intelligent systems that support\nhuman-in-the-loop applications for communicating ideas through sketching in art\nand design.\n