Frederick Kim and J.D. Zamfirescu-Pereira and Shm Almeda

EECS Department, University of California, Berkeley

Technical Report No. UCB/EECS-2023-248

December 1, 2023

http://www2.eecs.berkeley.edu/Pubs/TechRpts/2023/EECS-2023-248.pdf

Users of Text-to-Image (TTI) models like DALL•E and Stable Diffusion typically engage in a lot of iteration, exploring a design space, to achieve satisfactory outcomes. This design space’s input parameters consist of (1) prompt text spanning image content and style, and (2) stochastic (e.g., random seeds) and/or other opaque (e.g., classifier-free guidance) variables. Spreadsheets offer a natural interface for end-users to engage in design space exploration and rapid iteration in a “what- if" style. In this work, we present DreamSheets, a spreadsheet interface for creating images using TTI models. DreamSheets enables exploration of multiple input changes simultaneously, affording prompt-crafting using spreadsheet formula construction. Crucially, we also introduce a set of new functions that enable rapid exploration of the prompt input space, utilizing GPT-3 to generate context-relevant lists of prompt keyword options, new variations on existing prompts, and more. These functions enable DreamSheets users to rapidly explore the neighborhood around an initial prompt, leveraging the spreadsheet’s simultaneous display of these prompt-adjacent images. In a small formative study, we explore how the spreadsheet metaphor and these new functions impact participants in achieving and understanding artistic goals, concluding with some lessons learned for future designers of exploratory TTI-based systems.

Advisors: Björn Hartmann

---

BibTeX citation:

@mastersthesis{Kim:EECS-2023-248,
    Author= {Kim, Frederick and Zamfirescu-Pereira, J.D. and Almeda, Shm},
    Editor= {Hartmann, Björn},
    Title= {DreamSheets: Spreadsheets as Exploratory User Interface for Text-To-Image Models},
    School= {EECS Department, University of California, Berkeley},
    Year= {2023},
    Month= {Dec},
    Url= {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2023/EECS-2023-248.html},
    Number= {UCB/EECS-2023-248},
    Abstract= {Users of Text-to-Image (TTI) models like DALL•E and Stable Diffusion typically engage in a lot of iteration, exploring a design space, to achieve satisfactory outcomes. This design space’s input parameters consist of (1) prompt text spanning image content and style, and (2) stochastic (e.g., random seeds) and/or other opaque (e.g., classifier-free guidance) variables. Spreadsheets offer a natural interface for end-users to engage in design space exploration and rapid iteration in a “what- if" style. In this work, we present DreamSheets, a spreadsheet interface for creating images using TTI models. DreamSheets enables exploration of multiple input changes simultaneously, affording prompt-crafting using spreadsheet formula construction. Crucially, we also introduce a set of new functions that enable rapid exploration of the prompt input space, utilizing GPT-3 to generate context-relevant lists of prompt keyword options, new variations on existing prompts, and more. These functions enable DreamSheets users to rapidly explore the neighborhood around an initial prompt, leveraging the spreadsheet’s simultaneous display of these prompt-adjacent images. In a small formative study, we explore how the spreadsheet metaphor and these new functions impact participants in achieving and understanding artistic goals, concluding with some lessons learned for future designers of exploratory TTI-based systems.},
}

EndNote citation:

%0 Thesis
%A Kim, Frederick 
%A Zamfirescu-Pereira, J.D. 
%A Almeda, Shm 
%E Hartmann, Björn 
%T DreamSheets: Spreadsheets as Exploratory User Interface for Text-To-Image Models
%I EECS Department, University of California, Berkeley
%D 2023
%8 December 1
%@ UCB/EECS-2023-248
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2023/EECS-2023-248.html
%F Kim:EECS-2023-248