Minsuk Kahng

News

• Oct 2022 I started my job at Google in the Atlanta office.
• July 2022 DendroMap accepted for VIS'22. Check out this Twitter post.
• June 2022 I'll be joining Google's PAIR team in Atlanta as a Research Scientist later this year!
• Feb 2022 Serving as a Program Committee member for IEEE VIS.
• Jan 2022 New paper on video adaptation collaborated with KAIST accepted for CHI'22.
• Dec 2021 Thanks NAVER AI Lab for their gift to support our research collaboration!
• Oct 2021 New paper on explaining image classifiers by OSU XAI team accepted for NeurIPS'21.
• July 2021 Excited to be part of AgAID, a new NSF/USDA AI Institute ($20M funded).
• July 2021
  Two papers accepted for short presentations at IEEE VIS'21.
  - Contrastive Identification of Covariate Shift in Image Data
  - "Why did my AI agent lose?": Visual Analytics for Scaling Up AAR/AI
• June 2021 CNN Explainer (VIS'20 paper) is invited to present at ACM SIGGRAPH'21.
• Apr 2021 Honored to receive the 2021 Georgia Tech College of Computing Dissertation Award.
• Mar 2021 DARPA granted $600K for our Explainable AI research (co-PI w/ A. Fern, M. Burnett).

Research Interests

• Visual analytics for explaining errors in AI systems
• Iterative ML development and debugging by interacting with datasets
• Interactive analysis of bias and (un)fairness in ML datasets and large pretrained models
• Visual interfaces for exploring large image, text, and video datasets

Employment

• Google, Atlanta, GA Oct. 2022 - present
  Research Scientist, People+AI Research (PAIR) Team
• Oregon State University, Corvallis, OR 2019-2022
  Assistant Professor of Computer Science, School of Electrical Engineering and Computer Science
• Google, Cambridge, MA Summer 2017
  Software Engineering Intern, People+AI Research Team
• Facebook, Menlo Park, CA Summer 2016
  Research Intern, Applied ML Research Group
• Facebook, Menlo Park, CA Summer 2015
  Research Intern, Applied ML Research Group

Education

• Ph.D. in Computer Science, Georgia Institute of Technology, Atlanta, GA 2013-2019
  Thesis: Human-Centered AI through Scalable Visual Data Analytics
  Committee: Polo Chau, Sham Navathe, Alex Endert, Martin Wattenberg, Fernanda Viégas
• M.S. in Computer Science and Engineering, Seoul National University, South Korea 2009-2011
  Thesis: Context-Aware Recommendation using Learning-to-Rank (Advisor: Sang-goo Lee)
• B.S. in Electrical and Computer Engineering, Seoul National University, South Korea 2005-2009

Awards (selected)

• 2021 College of Computing Dissertation Award, Georgia Tech 2021
• Finalist, Facebook Research Award 2021
• ACM Trans. Interactive Intelligent Systems (TiiS) 2018 Best Paper, Honorable Mention 2020
• Google PhD Fellowship, Google AI 2018-2019
• Graduate TA of the Year in School of Computer Science, Georgia Tech 2018
• NSF Graduate Research Fellowship, National Science Foundation 2014-2017
• Best Paper Award, PhD Workshop at CIKM 2011
• National Scholarship for Science and Engineering, Korea Student Aid Foundation 2005-2009

Research Highlights

ActiVis
Visual Exploration of Industry-Scale ML Models
Collaborated with Facebook. Presented at IEEE VIS'17
DOI PDF Video Website

GAN Lab
Playing with Generative Models in Browser
Collaborated with Google Brain's PAIR. Presented at IEEE VIS'18 and VIS'20
DOI PDF Code Website

FairVis
Discovering Intersectional Bias in ML Results
Open-sourced; Presented at VIS'19
DOI PDF Blog Website

Publications (Latest & Greatest) (h-index: 20)

• 
  DendroMap: Visual Exploration of Large-Scale Image Datasets for Machine Learning with Treemaps.
  Donald Bertucci, Md Montaser Hamid, Yashwanthi Anand, Anita Ruangrotsakun, Delyar Tabatabai, Melissa Perez, Minsuk Kahng
  IEEE Transactions on Visualization and Computer Graphics, 29(1) (VIS'22), 2023.
  DOI PDF Twitter Post Demo
• Beyond Value: CheckList for Testing Inferences in Planning-Based RL.
  Kin-Ho Lam, Delyar Tabatabai, Jed Irvine, Donald Bertucci, Anita Ruangrotsakun, Minsuk Kahng, Alan Fern
  32nd International Conference on Automated Planning and Scheduling (ICAPS'22), 2022.
  PDF
• FitVid: Responsive and Flexible Video Content Adaptation.
  Jeongyeon Kim, Yubin Choi, Minsuk Kahng, Juho Kim
  ACM CHI Conference on Human Factors in Computing Systems (CHI'22), 2022.
  DOI PDF
• Finding AI's Faults with AAR/AI: An Empirical Study.
  Roli Khanna, Jonathan Dodge, Andrew Anderson, Rupika Dikkala, Jed Irvine, Zeyad Shureih, Kin-Ho Lam, Caleb R. Matthews, Zhengxian Lin, Minsuk Kahng, Alan Fern, Margaret Burnett
  ACM Transactions on Interactive Intelligent Systems, 12(1) (TiiS), 2022.
  DOI PDF
• One Explanation is Not Enough: Structured Attention Graphs for Image Classification.
  Vivswan Shitole, Li Fuxin, Minsuk Kahng, Prasad Tadepalli, Alan Fern
  35th Conference on Neural Information Processing Systems (NeurIPS'21), 2021.
  PDF arXiv
• 
  Contrastive Identification of Covariate Shift in Image Data
  Matthew L. Olson, Thuy-Vy Nguyen, Gaurav Dixit, Neale Ratzlaff, Weng-Keen Wong, Minsuk Kahng
  IEEE Visualization Conference (VIS'21), 2021.
  DOI PDF
• 
  "Why did my AI agent lose?": Visual Analytics for Scaling Up After-Action Review
  Delyar Tabatabai, Anita Ruangrotsakun, Jed Irvine, Jonathan Dodge, Zeyad Shureih, Kin-Ho Lam, Margaret Burnett, Alan Fern, Minsuk Kahng
  IEEE Visualization Conference (VIS'21), 2021.
  DOI PDF
• 
  CNN Explainer: Learning Convolutional Neural Networks with Interactive Visualization
  Zijie J. Wang, Robert Turko, Omar Shaikh, Haekyu Park, Nilaksh Das, Fred Hohman, Minsuk Kahng, Duen Horng (Polo) Chau
  IEEE Transactions on Visualization and Computer Graphics, 26(2) (VIS'20), 2021.
  DOI PDF Demo Video Code
• How Does Visualization Help People Learn Deep Learning? Evaluation of GAN Lab with Observational Study and Log Analysis
  Minsuk Kahng, Duen Horng (Polo) Chau
  IEEE Visualization Conference (VIS'20), 2020.
  PDF
• 
  FairVis: Visual Analytics for Discovering Intersectional Bias in Machine Learning
  Ángel Alexander Cabrera, Will Epperson, Fred Hohman, Minsuk Kahng, Jamie Morgenstern, Duen Horng (Polo) Chau
  IEEE Conference on Visual Analytics Science and Technology (VIS'19), 2019.
  DOI PDF arXiv Blog Code
  Featured in the Data Stories podcast (link)
• 
  GAN Lab: Understanding Complex Deep Generative Models using Interactive Visual Experimentation
  Minsuk Kahng, Nikhil Thorat, Duen Horng (Polo) Chau, Fernanda Viégas, Martin Wattenberg
  IEEE Transactions on Visualization and Computer Graphics, 25(1) (VIS'18), 2019.
  Open sourced with Google AI
  DOI PDF Slides Code Website
• Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers
  Fred Hohman, Minsuk Kahng, Robert Pienta, Duen Horng (Polo) Chau
  IEEE Transactions on Visualization and Computer Graphics, 25(8), 2019.
  DOI PDF Website Medium
  Most cited paper among all papers published in the journal in 2017-2021 (link)
• 
  ActiVis: Visual Exploration of Industry-Scale Deep Neural Network Models
  Minsuk Kahng, Pierre Y. Andrews, Aditya Kalro, Duen Horng (Polo) Chau
  IEEE Transactions on Visualization and Computer Graphics, 24(1) (VIS'17), 2018.
  Deployed on Facebook ML Platform; Invited to present at SIGGRAPH'18 as a top VIS paper
  DOI PDF Video Slides Website
• Chronodes: Interactive Multifocus Exploration of Event Sequences
  Peter J. Polack Jr., Shang-Tse Chen, Minsuk Kahng, Kaya De Barbaro, Rahul Basole, Moushumi Sharmin, Duen Horng (Polo) Chau
  ACM Transactions on Interactive Intelligent Systems (TiiS), 8(1), 2018.
  Best Paper, Honorable Mention
  DOI PDF
• FACETS: Adaptive Local Exploration of Large Graphs
  Robert Pienta, Minsuk Kahng, Zhiyuan Lin, Jilles Vreeken, Partha Talukdar, James Abello, Ganesh Parameswaran, Duen Horng (Polo) Chau
  SIAM International Conference on Data Mining (SDM'17), 2017.
  DOI PDF
• 
  Interactive Browsing and Navigation in Relational Databases
  Minsuk Kahng, Shamkant B. Navathe, John T. Stasko, Duen Horng (Polo) Chau
  Proceedings of the VLDB Endowment, 9(12) (VLDB'16), 2016.
  DOI PDF Slides
• 
  Visual Exploration of Machine Learning Results using Data Cube Analysis
  Minsuk Kahng, Dezhi Fang, Duen Horng (Polo) Chau
  Workshop on Human-In-the-Loop Data Analytics (HILDA at SIGMOD'16), 2016.
  Deployed on Facebook ML Platform
  DOI PDF Slides
• Understanding Variations in Pediatric Asthma Care Processes in the Emergency Department using Visual Analytics
  Rahul C. Basole, Mark Braunstein, Vikas Kumar, Hyunwoo Park, Minsuk Kahng, Duen Horng (Polo) Chau, Acar Tamersoy, Daniel A. Hirsh, Nicoleta Serban, James Bost, Burton Lesnick, Beth L. Schissel, Michael Thompson
  Journal of the American Medical Informatics Association, 22(2) (Special Issue on Visual Analytics in Healthcare), 2015.
  DOI
• 
  GLO-STIX: Graph-Level Operations for Specifying Techniques and Interactive eXploration
  Charles D. Stolper, Minsuk Kahng, Zhiyuan Lin, Florian Foerster, Aakash Goel, John Stasko, and Duen Horng (Polo) Chau
  IEEE Transactions on Visualization and Computer Graphics, 20(12) (InfoVis as part of VIS'14), 2014.
  DOI PDF Video
• PathRank: A Novel Node Ranking Measure on a Heterogeneous Graph for Recommender Systems
  Sangkeun Lee, Sungchan Park, Minsuk Kahng, Sang-goo Lee
  ACM Conference on Information and Knowledge Management (CIKM'12), 2012.
  DOI
• Exploiting Paths for Entity Search in RDF Graphs
  Minsuk Kahng, Sang-goo Lee
  ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'12 Poster), 2012.
  DOI PDF Poster
• Ranking Objects by Following Paths in Entity-Relationship Graphs
  Minsuk Kahng, Sangkeun Lee, Sang-goo Lee
  ACM Workshop for Ph.D. Students in Information and Knowledge Management (Ph.D. Workshop at CIKM'11), 2011.
  Best Paper Award
  DOI PDF Slides
• Ranking in Context-Aware Recommender Systems
  Minsuk Kahng, Sangkeun Lee, Sang-goo Lee
  International Conference on World Wide Web (WWW'11 Poster), 2011.
  DOI PDF Poster
• Random Walk based Entity Ranking on Graph for Multidimensional Recommendation
  Sangkeun Lee, Sang-il Song, Minsuk Kahng, Dongjoo Lee, Sang-goo Lee
  ACM Conference on Recommender Systems (RecSys'11), 2011.
  DOI PDF

Full list of papers

Teaching

• CS 499/549. Visual Analytics (Special Topic; Selected Topic in Data Science & Systems)
  Winter 2022
  This new course introduces "visual analytics", a field of study on combining interactive data visualization with automated analysis techniques for understanding and decision making from large, complex data. Students will learn how to design and build interactive data visualization interfaces for users to effectively explore and analyze data and discover insights. This course includes programming activities for developing web-based data visualization interfaces using JavaScript.
• CS 565. Human-Computer Interaction
  Spring 2022, Spring 2021, and Spring 2020
  This course provides students with basic principles of and research methods in Human-Computer Interaction (HCI). Students will learn how to design and prototype user interfaces and interactive systems, based on the needs of users, and how to evaluate such interfaces and systems rigorously.
• CS 539. Data Visualization for Machine Learning (Selected Topic in AI)
  Fall 2020
  This course introduces advanced state-of-the-art research on interactive data visualization for machine learning. Students will learn interactive data visualization methods and tools that are interpretable to complex ML models, scalable to large data, and usable to a variety of users (e.g., ML researchers, practitioners like ML engineers and data scientists, non-expert learners).

Student Advising

Graduate Students

• Eric Slyman, CS/AI PhD (co-advised with Stefan Lee), Fall 2021 - Summer 2022
• Yashwanthi Anand, CS MS/PhD, Fall 2021 - Spring 2022
• Montaser Hamid, CS PhD, Spring 2021 - Spring 2022
• Delyar Tabatabai, CS MS, Winter 2021 - Spring 2022
• Anita Ruangrotsakun, CS BS/MS (Accelerated Master's), Winter 2021 - Spring 2022
• Dayeon Oh, CS MS, Spring 2020 - Spring 2022 (Graduated)
• Roli Khanna, CS MS, Spring 2020 - Spring 2021 (Graduated; Now at Microsoft)

Undergraduate Students

• Donald Bertucci, CS, Winter 2020 - Summer 2022
• Melissa Perez, CS (Online), Fall 2021 - Spring 2022
• Anita Ruangrotsakun, CS BS/MS, Summer 2020 - Winter 2022
• Mark Ser, CS, Fall 2020 - Spring 2021 (OSU STEM Leader Program)
• Kristina Lee, CS, Fall 2020 - Winter 2021
• Thuy-Vy Nguyen, CS, Summer 2020 - Spring 2021 (Graduated; Now at Oracle)
• Junhyeok "Derek" Jeong, CS, Winter 2020 - Fall 2020

Grants & Funding

• NSF National AI Research Institute Grant & Gifts
  Title: USDA-NIFA Institute for Agricultural AI for Transforming Workforce and Decision Support
  Senior Personnel (Lead PI: Ananth Kalyanaraman)
  Total Amount: $20,000,000 (OSU: $6,500,000) (Period: 2021-26)
• DARPA Explainable Artificial Intelligence (XAI)
  Title: xACT: Explanation-Informed Acceptance Testing of Deep Adaptive Programs
  Co-PI (Lead PI: Alan Fern)
  Total Amount: $6,500,000 + $600,000 (Period: 2017-21, 2021-22)
• NAVER AI Lab
  Unrestricted Gift
  Amount: $100,000 for Year 1, 2021
• NSF Industry-University Collaboration Research Center on Pervasive Personalized Intelligence
  Project: Visual Analytics for Scalable AI Debugging
  Project PI (Site PI: Weng-Keen Wong): Collaboration with NEC and Intel
  Total Amount for OSU for Year 1: $64,000 (Period: 2021-present)
• Google Cloud Research Credit Credit
  Amount: $5,000, 2021
• Google PhD Fellowship Fellowship
  Full Tuition + $35,000 for 2 years, 2018-2019
• NSF Graduate Research Fellowship
  Full Tuition + $34,000 for 3 years, 2014-2017

Professional Service

Workshop Co-Organizer

• KDD 2018 Workshop on Interactive Data Exploration and Analytics (IDEA 2018)

Journal Co-Editor

• ACM Transactions on Interactive Intelligent Systems (TiiS), Special Issue Highlights of IUI 2019

Conference Organizing Committee: Webmaster and Web Designer

• WSDM 2016

Conference Program Committee

• IEEE VIS 2021-22
• IUI 2019-23
• AAAI 2021–22
• WSDM 2022 (Demo)
• IEEE VIS (Short Papers) 2020
• SDM 2020
• CIKM (Demo) 2019
• IUI (Poster and Demo) 2019

Workshop Program Committee

• Workshop on Human-In-the-Loop Data Analytics (HILDA at SIGMOD 2022)
• Workshop on Visualization Meets AI (at PacificVis 2020-21)
• Symposium on Visualization in Data Science (at VIS 2018–19)
• Workshop on Big Data Tools, Methods, and Use Cases for Innovative Scientific Discovery (at BigData 2019)
• Workshop on Visualization for AI Explainability (at VIS 2018)
• KDD Workshop on Interactive Data Exploration and Analytics (IDEA 2016–17)
• Workshop on Visual Analytics for Deep Learning (at VIS 2017)

Journal Reviewer

• IEEE Transactions on Visualization and Computer Graphics (TVCG) (2019, 2021-22)
• ACM Transactions on Interactive Intelligent Systems (TiiS) (2020-21)
• ACM Transactions on Intelligent Systems and Technology (TIST) (2020, 2022)
• Distill (2019)
• ACM Transactions on Computer-Human Interaction (TOCHI) (2015, 2018)
• Expert Systems with Applications (2015)

Conference Reviewer

• CHI 2014, 2017–19, 2021-22
• CSCW 2020
• VIS 2018–20
• EuroVis 2018
• SDM 2014, 2016–17
• KDD 2014–16
• IUI 2016
• RecSys 2016
• SIGMOD 2013
• DASFAA 2011