Method for Exploring Generative Adversarial Networks (GANs) via Automatically Generated Image Galleries | Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (2024)

research-article

Authors: Enhao Zhang and Nikola Banovic

CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

May 2021

Article No.: 76, Pages 1 - 15

Published: 07 May 2021 Publication History

  • 18citation
  • 1,286
  • Downloads

Metrics

Total Citations18Total Downloads1,286

Last 12 Months170

Last 6 weeks7

  • Get Citation Alerts

    New Citation Alert added!

    This alert has been successfully added and will be sent to:

    You will be notified whenever a record that you have chosen has been cited.

    To manage your alert preferences, click on the button below.

    Manage my Alerts

    New Citation Alert!

    Please log in to your account

  • Get Access

      • Get Access
      • References
      • Media
      • Tables
      • Share

    Abstract

    Generative Adversarial Networks (GANs) can automatically generate quality images from learned model parameters. However, it remains challenging to explore and objectively assess the quality of all possible images generated using a GAN. Currently, model creators evaluate their GANs via tedious visual examination of generated images sampled from narrow prior probability distributions on model parameters. Here, we introduce an interactive method to explore and sample quality images from GANs. Our first two user studies showed that participants can use the tool to explore a GAN and select quality images. Our third user study showed that images sampled from a posterior probability distribution using a Markov Chain Monte Carlo (MCMC) method on parameters of images collected in our first study resulted in on average higher quality and more diverse images than existing baselines. Our work enables principled qualitative GAN exploration and evaluation.

    Supplementary Material

    MP4 File (3411764.3445714_videofigure.mp4)

    Supplemental video

    • Download
    • 59.16 MB
    MP4 File (3411764.3445714_videopreview.mp4)

    Preview video

    • Download
    • 12.36 MB

    References

    [1]

    Amazon. 2020. Amazon Mechanical Turk. https://www.mturk.com/

    [2]

    Christophe Andrieu, Nando DeFreitas, Arnaud Doucet, and MichaelI Jordan. 2003. An introduction to MCMC for machine learning. Machine learning 50, 1-2 (2003), 5–43.

    [3]

    David Bau, Jun-Yan Zhu, Hendrik Strobelt, Bolei Zhou, JoshuaB. Tenenbaum, WilliamT. Freeman, and Antonio Torralba. 2019. GAN Dissection: Visualizing and Understanding Generative Adversarial Networks. In Proceedings of the International Conference on Learning Representations (ICLR).

    [4]

    Ali Borji. 2019. Pros and cons of GAN evaluation measures. Computer Vision and Image Understanding 179 (2019), 41 – 65. https://doi.org/10.1016/j.cviu.2018.10.009

    Digital Library

    [5]

    Eric Brochu, VladM Cora, and Nando DeFreitas. 2010. A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. arXiv preprint arXiv:1012.2599(2010).

    [6]

    Andrew Brock, Jeff Donahue, and Karen Simonyan. 2018. Large Scale GAN Training for High Fidelity Natural Image Synthesis. arxiv:1809.11096[cs.LG]

    [7]

    Andrew Brock, Jeff Donahue, and Karen Simonyan. 2019. Large Scale GAN Training for High Fidelity Natural Image Synthesis. In International Conference on Learning Representations. https://openreview.net/forum?id=B1xsqj09Fm

    [8]

    Andrew Brock, Theodore Lim, JamesM Ritchie, and Nick Weston. 2016. Neural photo editing with introspective adversarial networks. arXiv preprint arXiv:1609.07093(2016).

    [9]

    Steve Brooks, Andrew Gelman, Galin Jones, and Xiao-Li Meng(Eds.). 2011. Handbook of Markov Chain Monte Carlo. Chapman and Hall.

    [10]

    Wengling Chen and James Hays. 2018. SketchyGAN: Towards Diverse and Realistic Sketch to Image Synthesis. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

    [11]

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition. Ieee, 248–255.

    [12]

    Brochu Eric, NandoD Freitas, and Abhijeet Ghosh. 2008. Active preference learning with discrete choice data. In Advances in neural information processing systems. 409–416.

    [13]

    Edoardo Giacomello, PierLuca Lanzi, and Daniele Loiacono. 2019. Searching the Latent Space of a Generative Adversarial Network to Generate DOOM Levels. In 2019 IEEE Conference on Games (CoG). 1–8. https://doi.org/10.1109/CIG.2019.8848011

    Digital Library

    [14]

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative Adversarial Nets. In Advances in Neural Information Processing Systems 27, Z.Ghahramani, M.Welling, C.Cortes, N.D. Lawrence, and K.Q. Weinberger(Eds.). Curran Associates, Inc., 2672–2680. http://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf

    [15]

    Aaron Hertzmann. 2020. Visual Indeterminacy in GAN Art. Leonardo 53, 4 (2020), 424–428. https://doi.org/10.1162/leon_a_01930arXiv:https://doi.org/10.1162/leon_a_01930

    [16]

    MatthewD Hoffman and Andrew Gelman. 2014. The No-U-Turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo.J. Mach. Learn. Res. 15, 1 (2014), 1593–1623.

    [17]

    Xun Huang, Ming-Yu Liu, Serge Belongie, and Jan Kautz. 2018. Multimodal Unsupervised Image-to-image Translation. In Proceedings of the European Conference on Computer Vision (ECCV).

    Digital Library

    [18]

    Erik Härkönen, Aaron Hertzmann, Jaakko Lehtinen, and Sylvain Paris. 2020. GANSpace: Discovering Interpretable GAN Controls. In Advances in Neural Information Processing Systems, Vol.33. Curran Associates, Inc.https://proceedings.neurips.cc/paper/2020/file/6fe43269967adbb64ec6149852b5cc3e-Paper.pdf

    [19]

    Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and AlexeiA. Efros. 2017. Image-To-Image Translation With Conditional Adversarial Networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

    [20]

    Mario Klingemann. 2020. Trapping the Accident. In PhotographyDigitalPainting: Expanding Medium Interconnectivity in Contemporary Visual Art Practices(first ed.), Carl Robinson (Ed.). Cambridge Scholars Publishing, 77–98.

    [21]

    Yuki Koyama, Issei Sato, and Masataka Goto. 2020. Sequential Gallery for Interactive Visual Design Optimization. ACM Trans. Graph. 39, 4, Article 88 (July 2020), 12pages. https://doi.org/10.1145/3386569.3392444

    Digital Library

    [22]

    Yuki Koyama, Issei Sato, Daisuke Sakamoto, and Takeo Igarashi. 2017. Sequential Line Search for Efficient Visual Design Optimization by Crowds. ACM Trans. Graph. 36, 4, Article 48 (July 2017), 11pages. https://doi.org/10.1145/3072959.3073598

    Digital Library

    [23]

    J. Marks, B. Andalman, P.A. Beardsley, W. Freeman, S. Gibson, J. Hodgins, T. Kang, B. Mirtich, H. Pfister, W. Ruml, K. Ryall, J. Seims, and S. Shieber. 1997. Design Galleries: A General Approach to Setting Parameters for Computer Graphics and Animation. In Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques(SIGGRAPH ’97). ACM Press/Addison-Wesley Publishing Co., USA, 389–400. https://doi.org/10.1145/258734.258887

    Digital Library

    [24]

    Jakob Nielsen and Rolf Molich. 1990. Heuristic Evaluation of User Interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Seattle, Washington, USA) (CHI ’90). Association for Computing Machinery, New York, NY, USA, 249–256. https://doi.org/10.1145/97243.97281

    Digital Library

    [25]

    Taesung Park, Ming-Yu Liu, Ting-Chun Wang, and Jun-Yan Zhu. 2019. Semantic Image Synthesis With Spatially-Adaptive Normalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

    [26]

    Alec Radford, Luke Metz, and Soumith Chintala. 2015. Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks. arxiv:1511.06434[cs.LG]

    [27]

    Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, Xi Chen, and Xi Chen. 2016. Improved Techniques for Training GANs. In Advances in Neural Information Processing Systems 29, D.D. Lee, M.Sugiyama, U.V. Luxburg, I.Guyon, and R.Garnett (Eds.). Curran Associates, Inc., 2234–2242. http://papers.nips.cc/paper/6125-improved-techniques-for-training-gans.pdf

    [28]

    John Salvatier, ThomasV Wiecki, and Christopher Fonnesbeck. 2016. Probabilistic programming in Python using PyMC3. PeerJ Computer Science 2(2016), e55.

    [29]

    Helena Sarin. 2019. The Books of GANesis: Divine Comedy in Tangled Representations. Helena Sarin.

    [30]

    Jacob Schrum, Jake Gutierrez, Vanessa Volz, Jialin Liu, Simon Lucas, and Sebastian Risi. 2020. Interactive Evolution and Exploration within Latent Level-Design Space of Generative Adversarial Networks. In Proceedings of the 2020 Genetic and Evolutionary Computation Conference (Cancún, Mexico) (GECCO ’20). Association for Computing Machinery, New York, NY, USA, 148–156. https://doi.org/10.1145/3377930.3389821

    Digital Library

    [31]

    Ben Shneiderman. 2007. Creativity Support Tools: Accelerating Discovery and Innovation. Commun. ACM 50, 12 (Dec. 2007), 20–32. https://doi.org/10.1145/1323688.1323689

    Digital Library

    [32]

    JacobO. Wobbrock, Leah Findlater, Darren Gergle, and JamesJ. Higgins. 2011. The Aligned Rank Transform for Nonparametric Factorial Analyses Using Only Anova Procedures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Vancouver, BC, Canada) (CHI ’11). Association for Computing Machinery, New York, NY, USA, 143–146. https://doi.org/10.1145/1978942.1978963

    Digital Library

    [33]

    Huikai Wu, Shuai Zheng, Junge Zhang, and Kaiqi Huang. 2019. GP-GAN: Towards Realistic High-Resolution Image Blending. In Proceedings of the 27th ACM International Conference on Multimedia (Nice, France) (MM ’19). Association for Computing Machinery, New York, NY, USA, 2487–2495. https://doi.org/10.1145/3343031.3350944

    Digital Library

    [34]

    Sharon Zhou, Mitchell Gordon, Ranjay Krishna, Austin Narcomey, LiF Fei-Fei, and Michael Bernstein. 2019. HYPE: A Benchmark for Human eYe Perceptual Evaluation of Generative Models. In Advances in Neural Information Processing Systems 32, H.Wallach, H.Larochelle, A.Beygelzimer, F.d'Alché-Buc, E.Fox, and R.Garnett (Eds.). Curran Associates, Inc., 3449–3461. http://papers.nips.cc/paper/8605-hype-a-benchmark-for-human-eye-perceptual-evaluation-of-generative-models.pdf

    [35]

    Jun-Yan Zhu, Philipp Krähenbühl, Eli Shechtman, and AlexeiA. Efros. 2016. Generative Visual Manipulation on the Natural Image Manifold. In Computer Vision – ECCV 2016, Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling(Eds.). Springer International Publishing, Cham, 597–613.

    Cited By

    View all

    • Vincenzi BStumpf STaylor ANakao Y(2024)Lay User Involvement in Developing Human-centric Responsible AI Systems: When and How?ACM Journal on Responsible Computing10.1145/36525921:2(1-25)Online publication date: 15-Mar-2024

      https://dl.acm.org/doi/10.1145/3652592

    • Choi DHong SPark JChung JKim J(2024)CreativeConnect: Supporting Reference Recombination for Graphic Design Ideation with Generative AIProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642794(1-25)Online publication date: 11-May-2024

      https://dl.acm.org/doi/10.1145/3613904.3642794

    • Xiao SWang LMa XZeng W(2024)TypeDance: Creating Semantic Typographic Logos from Image through Personalized GenerationProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642185(1-18)Online publication date: 11-May-2024

      https://dl.acm.org/doi/10.1145/3613904.3642185

    • Show More Cited By

    Index Terms

    1. Method for Exploring Generative Adversarial Networks (GANs) via Automatically Generated Image Galleries

      1. Computing methodologies

        1. Artificial intelligence

          1. Computer vision

          2. Computer graphics

            1. Machine learning

          Index terms have been assigned to the content through auto-classification.

          Recommendations

          • Image Denoising via Generative Adversarial Networks with Detail Loss

            ICISS '19: Proceedings of the 2nd International Conference on Information Science and Systems

            Image denoising is a challenging task which aims to remove additional noise and preserve all useful information. Many existing image denoising algorithms focus on improving the typical object measure, peak signal-to-noise ratio (PSNR), and take the mean ...

            Read More

          • Image Dehazing Via Cycle Generative Adversarial Network

            AISS '21: Proceedings of the 3rd International Conference on Advanced Information Science and System

            Recovering a clear image from single hazy image has been widely investigated in recent researches. Due to the lack of the real hazed image dataset, most studies use artificially synthesized dataset to train the models. Nonetheless, the real word foggy ...

            Read More

          • A Method forFace Image Inpainting Based onAutoencoder andGenerative Adversarial Network

            Image and Video Technology

            Abstract

            Face image inpainting has great value in the fields of computer vision and digital image processing. In this paper, we propose a face image inpainting method based on autoencoder and Generative Adversarial Network (GAN). The neural network for ...

            Read More

          Comments

          Information & Contributors

          Information

          Published In

          Method for Exploring Generative Adversarial Networks (GANs) via Automatically Generated Image Galleries | Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (3)

          CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

          May 2021

          10862 pages

          ISBN:9781450380966

          DOI:10.1145/3411764

          • General Chairs:
          • Yoshifumi Kitamura

            Tohoku University, Japan

            ,
          • Aaron Quigley

            University of New South Wales, Australia

            ,
          • Program Chairs:
          • Katherine Isbister

            University of California Santa Cruz, USA

            ,
          • Takeo Igarashi

            The University of Tokyo, Japan

            ,
          • Publications Chairs:
          • Pernille Bjørn

            University of Copenhagen, Denmark

            ,
          • Steven Drucker

            Microsoft Research, USA

          Copyright © 2021 ACM.

          Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [emailprotected].

          Sponsors

          • SIGCHI: ACM Special Interest Group on Computer-Human Interaction

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          Published: 07 May 2021

          Permissions

          Request permissions for this article.

          Check for updates

          Author Tags

          1. Interactive model exploration
          2. qualitative model validation.

          Qualifiers

          • Research-article
          • Research
          • Refereed limited

          Conference

          CHI '21

          Sponsor:

          • SIGCHI

          Acceptance Rates

          Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

          Upcoming Conference

          CHI PLAY '24

          • Sponsor:
          • sigchi

          The Annual Symposium on Computer-Human Interaction in Play

          October 14 - 17, 2024

          Tampere , Finland

          Contributors

          Method for Exploring Generative Adversarial Networks (GANs) via Automatically Generated Image Galleries | Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (10)

          Other Metrics

          View Article Metrics

          Bibliometrics & Citations

          Bibliometrics

          Article Metrics

          • 18

            Total Citations

            View Citations
          • 1,286

            Total Downloads

          • Downloads (Last 12 months)170
          • Downloads (Last 6 weeks)7

          Other Metrics

          View Author Metrics

          Citations

          Cited By

          View all

          • Vincenzi BStumpf STaylor ANakao Y(2024)Lay User Involvement in Developing Human-centric Responsible AI Systems: When and How?ACM Journal on Responsible Computing10.1145/36525921:2(1-25)Online publication date: 15-Mar-2024

            https://dl.acm.org/doi/10.1145/3652592

          • Choi DHong SPark JChung JKim J(2024)CreativeConnect: Supporting Reference Recombination for Graphic Design Ideation with Generative AIProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642794(1-25)Online publication date: 11-May-2024

            https://dl.acm.org/doi/10.1145/3613904.3642794

          • Xiao SWang LMa XZeng W(2024)TypeDance: Creating Semantic Typographic Logos from Image through Personalized GenerationProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642185(1-18)Online publication date: 11-May-2024

            https://dl.acm.org/doi/10.1145/3613904.3642185

          • Pang RSanty SJust RReinecke K(2024)BLIP: Facilitating the Exploration of Undesirable Consequences of Digital TechnologiesProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642054(1-18)Online publication date: 11-May-2024

            https://dl.acm.org/doi/10.1145/3613904.3642054

          • Kim TLee YChang MKim J(2023)Cells, Generators, and Lenses: Design Framework for Object-Oriented Interaction with Large Language ModelsProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606833(1-18)Online publication date: 29-Oct-2023

            https://dl.acm.org/doi/10.1145/3586183.3606833

          • Chung JAdar E(2023)PromptPaint: Steering Text-to-Image Generation Through Paint Medium-like InteractionsProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606777(1-17)Online publication date: 29-Oct-2023

            https://dl.acm.org/doi/10.1145/3586183.3606777

          • Brachman MPan QDo HDugan CChaudhary AJohnson JRai PChakraborti TGschwind TLaredo JMiksovic CScotton PTalamadupula KThomas G(2023)Follow the Successful Herd: Towards Explanations for Improved Use and Mental Models of Natural Language SystemsProceedings of the 28th International Conference on Intelligent User Interfaces10.1145/3581641.3584088(220-239)Online publication date: 27-Mar-2023

            https://dl.acm.org/doi/10.1145/3581641.3584088

          • Ko HPark GJeon HJo JKim JSeo J(2023)Large-scale Text-to-Image Generation Models for Visual Artists’ Creative WorksProceedings of the 28th International Conference on Intelligent User Interfaces10.1145/3581641.3584078(919-933)Online publication date: 27-Mar-2023

            https://dl.acm.org/doi/10.1145/3581641.3584078

          • Antar AKratz ABanovic N(2023)Behavior Modeling Approach for Forecasting Physical Functioning of People with Multiple SclerosisProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35808877:1(1-29)Online publication date: 28-Mar-2023

            https://dl.acm.org/doi/10.1145/3580887

          • Banovic NYang ZRamesh ALiu A(2023)Being Trustworthy is Not Enough: How Untrustworthy Artificial Intelligence (AI) Can Deceive the End-Users and Gain Their TrustProceedings of the ACM on Human-Computer Interaction10.1145/35794607:CSCW1(1-17)Online publication date: 16-Apr-2023

            https://dl.acm.org/doi/10.1145/3579460

          • Show More Cited By

          View Options

          Get Access

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          Get this Publication

          View options

          PDF

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader

          HTML Format

          View this article in HTML Format.

          HTML Format

          Media

          Figures

          Other

          Tables

          Method for Exploring Generative Adversarial Networks (GANs) via Automatically Generated Image Galleries | Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (2024)
          Top Articles
          Number one song in USA on my 45th birthday was Waterfalls by TLC. Find the US #1 song on every birthday at MyBirthdayHits.com/us/
          Forest Green Park Cemetery | Morganville, New Jersey
          Live Basketball Scores Flashscore
          Mountain Dew Bennington Pontoon
          South Park Season 26 Kisscartoon
          DEA closing 2 offices in China even as the agency struggles to stem flow of fentanyl chemicals
          Jesus Calling December 1 2022
          Coffman Memorial Union | U of M Bookstores
          Jennette Mccurdy And Joe Tmz Photos
          The Best Classes in WoW War Within - Best Class in 11.0.2 | Dving Guides
          T&G Pallet Liquidation
          1TamilMV.prof: Exploring the latest in Tamil entertainment - Ninewall
          Mivf Mdcalc
          Devourer Of Gods Resprite
          Truist Drive Through Hours
          Best Restaurants Ventnor
          The Binding of Isaac
          Nhl Wikia
          Gemita Alvarez Desnuda
          DBZ Dokkan Battle Full-Power Tier List [All Cards Ranked]
          Lcwc 911 Live Incident List Live Status
          97226 Zip Code
          Kringloopwinkel Second Sale Roosendaal - Leemstraat 4e
          Craigslist Maryland Trucks - By Owner
          Kohls Lufkin Tx
          Sessional Dates U Of T
          The Eight of Cups Tarot Card Meaning - The Ultimate Guide
          Cal State Fullerton Titan Online
          Wku Lpn To Rn
          Ascensionpress Com Login
          Visit the UK as a Standard Visitor
          Bursar.okstate.edu
          Otis Offender Michigan
          R3Vlimited Forum
          Of An Age Showtimes Near Alamo Drafthouse Sloans Lake
          Whas Golf Card
          Closest 24 Hour Walmart
          Enjoy4Fun Uno
          Nsav Investorshub
          Doordash Promo Code Generator
          Craigs List Hartford
          At Home Hourly Pay
          Myrtle Beach Craigs List
          Login
          Iman Fashion Clearance
          Samsung 9C8
          Actress Zazie Crossword Clue
          Take Me To The Closest Ups
          Upcoming Live Online Auctions - Online Hunting Auctions
          Minute Clinic Mooresville Nc
          Strange World Showtimes Near Atlas Cinemas Great Lakes Stadium 16
          Costco Gas Price Fort Lauderdale
          Latest Posts
          Article information

          Author: Rubie Ullrich

          Last Updated:

          Views: 6151

          Rating: 4.1 / 5 (52 voted)

          Reviews: 91% of readers found this page helpful

          Author information

          Name: Rubie Ullrich

          Birthday: 1998-02-02

          Address: 743 Stoltenberg Center, Genovevaville, NJ 59925-3119

          Phone: +2202978377583

          Job: Administration Engineer

          Hobby: Surfing, Sailing, Listening to music, Web surfing, Kitesurfing, Geocaching, Backpacking

          Introduction: My name is Rubie Ullrich, I am a enthusiastic, perfect, tender, vivacious, talented, famous, delightful person who loves writing and wants to share my knowledge and understanding with you.