Hi, I'm an Assistant Professor in the School of Electrical Engineering and Computer Science at Oregon State University, where I lead the Data Interaction and Visualization (DIV) Lab.
My research focuses on building novel visual analytics tools for people to explore, interpret, and interact with machine learning systems and large datasets. In doing this, I combine human-centered interactive approaches and data-driven scalable techniques, by using methods in data visualization, Explainable AI, human-computer interaction (HCI), and databases.
I earned a PhD in computer science from Georgia Tech with a Dissertation Award. My work led to deployed technologies (e.g., ActiVis for Facebook) and open-sourced tools (e.g., GAN Lab developed with Google). My research has been supported by NSF, DARPA, Google, and more.
Research InterestsMy students and I are currently working on research in the following topics:
- Visual analytics for explaining errors in AI systems
- Iterative ML development and debugging by interacting with datasets
- Interactive analysis of bias and (un)fairness in ML datasets and large pretrained models
- Visual interfaces for exploring large image, text, and video datasets
- Feb 2022 Serving as a Program Committee member for IEEE VIS again this year.
- Feb 2022 New paper "Beyond Value: CheckList for Testing Inferences in Planning-Based RL" accepted.
- Dec 2021 Thanks NAVER AI Lab for their gift to support our research collaboration!
- Nov 2021 New paper on video adaptation collaborated with KAIST conditionally accepted for CHI'22.
- Oct 2021 New paper on explaining image classifiers by OSU XAI team accepted for NeurIPS'21.
- July 2021 Excited to be part of AgAID, a new NSF/USDA AI Institute ($20M funded).
July 2021Two papers accepted for short presentations at IEEE VIS'21.
- Contrastive Identification of Covariate Shift in Image Data
- "Why did my AI agent lose?": Visual Analytics for Scaling Up AAR/AI
- June 2021 CNN Explainer (VIS'20 paper) is invited to present at ACM SIGGRAPH'21.
- Apr 2021 Honored to receive the 2021 Georgia Tech College of Computing Dissertation Award.
- Mar 2021 DARPA granted $600K for our Explainable AI research (co-PI w/ A. Fern, M. Burnett).
Ph.D. in Computer Science,
Georgia Institute of Technology, USA
Thesis: Human-Centered AI through Scalable Visual Data Analytics
Committee: Polo Chau, Sham Navathe, Alex Endert, Martin Wattenberg, Fernanda Viégas
M.S. in Computer Science and Engineering,
Seoul National University, South Korea
Thesis: Context-Aware Recommendation using Learning-to-Rank (Advisor: Sang-goo Lee)
- B.S. in Electrical and Computer Engineering, Seoul National University, South Korea 2005-2009
Industry Research Experience
- Google, Software Engineering Intern at Google Brain's People+AI Research Group Summer 2017
- Facebook, Research Intern at Applied ML Research Group Summer 2016
- Facebook, Research Intern at Applied ML Research Group Summer 2015
- 2021 College of Computing Dissertation Award, Georgia Tech 2021
- Finalist, Facebook Research Award 2021
- ACM Trans. Interactive Intelligent Systems (TiiS) 2018 Best Paper, Honorable Mention 2020
- Google PhD Fellowship, Google AI 2018-2019
- Graduate TA of the Year in School of Computer Science, Georgia Tech 2018
- NSF Graduate Research Fellowship, National Science Foundation 2014-2017
- Best Paper Award, PhD Workshop at CIKM 2011
- National Scholarship for Science and Engineering, Korea Student Aid Foundation 2005-2009
Publications (Latest & Greatest) (h-index: 19)
Beyond Value: CheckList for Testing Inferences in Planning-Based RL.
32nd International Conference on Automated Planning and Scheduling (ICAPS'22), 2022.
FitVid: Responsive and Flexible Video Content Adaptation.
ACM CHI Conference on Human Factors in Computing Systems (CHI'22), 2022.
Finding AI's Faults with AAR/AI: An Empirical Study.
ACM Transactions on Interactive Intelligent Systems, 12(1) (TiiS), 2022.
One Explanation is Not Enough: Structured Attention Graphs for Image Classification.
35th Conference on Neural Information Processing Systems (NeurIPS'21), 2021.
Contrastive Identification of Covariate Shift in Image Data
IEEE Visualization Conference (VIS'21), 2021.
"Why did my AI agent lose?": Visual Analytics for Scaling Up After-Action Review
IEEE Visualization Conference (VIS'21), 2021.
CNN Explainer: Learning Convolutional Neural Networks with Interactive Visualization
IEEE Transactions on Visualization and Computer Graphics, 26(2) (VIS'20), 2021.
DOI PDF Demo Video Code
How Does Visualization Help People Learn Deep Learning? Evaluation of GAN Lab with Observational Study and Log Analysis
IEEE Visualization Conference (VIS'20), 2020.
FairVis: Visual Analytics for Discovering Intersectional Bias in Machine Learning
IEEE Conference on Visual Analytics Science and Technology (VIS'19), 2019.
DOI PDF arXiv Blog Code
Featured in the Data Stories podcast (link)
GAN Lab: Understanding Complex Deep Generative Models using Interactive Visual Experimentation
IEEE Transactions on Visualization and Computer Graphics, 25(1) (VIS'18), 2019.
Open sourced with Google AI
DOI PDF Slides Code Website
Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers
IEEE Transactions on Visualization and Computer Graphics, 25(8), 2019.
DOI PDF Website Medium
Most cited paper among all papers published in the journal in 2017-2021 (link)
ActiVis: Visual Exploration of Industry-Scale Deep Neural Network Models
IEEE Transactions on Visualization and Computer Graphics, 24(1) (VIS'17), 2018.
Deployed on Facebook ML Platform; Invited to present at SIGGRAPH'18 as a top VIS paper
DOI PDF Video Slides Website
Chronodes: Interactive Multifocus Exploration of Event Sequences
ACM Transactions on Interactive Intelligent Systems (TiiS), 8(1), 2018.
Best Paper, Honorable Mention
FACETS: Adaptive Local Exploration of Large Graphs
SIAM International Conference on Data Mining (SDM'17), 2017.
Interactive Browsing and Navigation in Relational Databases
Proceedings of the VLDB Endowment, 9(12) (VLDB'16), 2016.
DOI PDF Slides
Visual Exploration of Machine Learning Results using Data Cube Analysis
Workshop on Human-In-the-Loop Data Analytics (HILDA at SIGMOD'16), 2016.
Deployed on Facebook ML Platform
DOI PDF Slides
Understanding Variations in Pediatric Asthma Care Processes in the Emergency Department using Visual Analytics
Journal of the American Medical Informatics Association, 22(2) (Special Issue on Visual Analytics in Healthcare), 2015.
GLO-STIX: Graph-Level Operations for Specifying Techniques and Interactive eXploration
IEEE Transactions on Visualization and Computer Graphics, 20(12) (InfoVis as part of VIS'14), 2014.
DOI PDF Video
PathRank: A Novel Node Ranking Measure on a Heterogeneous Graph for Recommender Systems
ACM Conference on Information and Knowledge Management (CIKM'12), 2012.
Exploiting Paths for Entity Search in RDF Graphs
ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'12 Poster), 2012.
DOI PDF Poster
Ranking Objects by Following Paths in Entity-Relationship Graphs
ACM Workshop for Ph.D. Students in Information and Knowledge Management (Ph.D. Workshop at CIKM'11), 2011.
Best Paper Award
DOI PDF Slides
Ranking in Context-Aware Recommender Systems
International Conference on World Wide Web (WWW'11 Poster), 2011.
DOI PDF Poster
Random Walk based Entity Ranking on Graph for Multidimensional Recommendation
ACM Conference on Recommender Systems (RecSys'11), 2011.
CS 499/549. Visual Analytics (Special Topic; Selected Topic in Data Science & Systems)
CS 565. Human-Computer Interaction
Spring 2022, Spring 2021, and Spring 2020
This course provides students with basic principles of and research methods in Human-Computer Interaction (HCI). Students will learn how to design and prototype user interfaces and interactive systems, based on the needs of users, and how to evaluate such interfaces and systems rigorously.
CS 539. Data Visualization for Machine Learning (Selected Topic in AI)
This course introduces advanced state-of-the-art research on interactive data visualization for machine learning. Students will learn how to design and develop interactive data visualization methods and tools that are interpretable to complex ML models (e.g., deep learning models), scalable to large data, and usable to a variety of users (e.g., ML researchers, practitioners like ML engineers and data scientists, non-expert learners).
- Eric Slyman, CS/AI PhD (co-advised by Stefan Lee), Fall 2021 - present
- Yashwanthi Anand, CS MS/PhD, Fall 2021 - present
- Montaser Hamid, CS PhD, Spring 2021 - present
- Delyar Tabatabai, CS MS, Winter 2021 - present
- Kin-Ho Lam, CS MS (co-advised with Alan Fern), Fall 2020 - present
- Dayeon Oh, CS MS, Spring 2020 - present
CS MS, Spring 2020 - Spring 2021 (Graduated; Now at Microsoft)
Thesis: Assessing and Finding Faults in AI: Two Empirical Studies
- Melissa Perez, CS (Online), Fall 2021 - present
- Donald Bertucci, CS, Winter 2020 - present
- Anita Ruangrotsakun, CS BS/MS (AMP), Summer 2020 - present
- Mark Ser, CS, Fall 2020 - Spring 2021 (OSU STEM Leader Program)
- Kristina Lee, CS, Fall 2020 - Winter 2021
- Thuy-Vy Nguyen, CS, Summer 2020 - Spring 2021 (Graduated; Now at Oracle)
- Junhyeok "Derek" Jeong, CS, Winter 2020 - Fall 2020
Grants & Funding
NSF National AI Research Institute
Grant & Gifts
Title: USDA-NIFA Institute for Agricultural AI for Transforming Workforce and Decision Support
Senior Personnel (Lead PI: Ananth Kalyanaraman)
Total Amount: $20,000,000 (OSU: $6,500,000) (Period: 2021-26)
DARPA Explainable Artificial Intelligence (XAI)
Title: xACT: Explanation-Informed Acceptance Testing of Deep Adaptive Programs
Co-PI (Lead PI: Alan Fern)
Total Amount: $6,500,000 + $600,000 (Period: 2017-21, 2021-22)
NAVER AI Lab
Amount: $100,000 for Year 1, 2021
NSF Industry-University Collaboration Research Center on Pervasive Personalized Intelligence
Project: Visual Analytics for Scalable AI Debugging
Project PI (Site PI: Weng-Keen Wong): Collaboration with NEC and Intel
Total Amount for OSU for Year 1: $64,000 (Period: 2021-present)
Google Cloud Research Credit
Amount: $5,000, 2021
Google PhD Fellowship
Full Tuition + $35,000 for 2 years, 2018-2019
NSF Graduate Research Fellowship
Full Tuition + $34,000 for 3 years, 2014-2017
- KDD 2018 Workshop on Interactive Data Exploration and Analytics (IDEA 2018)
- ACM Transactions on Interactive Intelligent Systems (TiiS), Special Issue Highlights of IUI 2019
Conference Organizing Committee: Webmaster and Web Designer
- WSDM 2016
Conference Program Committee
- IEEE VIS 2021-22
- IUI 2019-22
- AAAI 2021–22
- IEEE VIS (Short Papers) 2020
- SDM 2020
- CIKM (Demo) 2019
- IUI (Poster and Demo) 2019
Workshop Program Committee
- Workshop on Human-In-the-Loop Data Analytics (HILDA at SIGMOD 2022)
- Workshop on Visualization Meets AI (at PacificVis 2020-21)
- Symposium on Visualization in Data Science (at VIS 2018–19)
- Workshop on Big Data Tools, Methods, and Use Cases for Innovative Scientific Discovery (at BigData 2019)
- Workshop on Visualization for AI Explainability (at VIS 2018)
- KDD Workshop on Interactive Data Exploration and Analytics (IDEA 2016–17)
- Workshop on Visual Analytics for Deep Learning (at VIS 2017)
- IEEE Transactions on Visualization and Computer Graphics (TVCG) (2019, 2021-22)
- ACM Transactions on Interactive Intelligent Systems (TiiS) (2020-21)
- ACM Transactions on Intelligent Systems and Technology (TIST) (2020, 2022)
- Distill (2019)
- ACM Transactions on Computer-Human Interaction (TOCHI) (2015, 2018)
- Expert Systems with Applications (2015)
- CHI 2014, 2017–19, 2021-22
- CSCW 2020
- VIS 2018–20
- EuroVis 2018
- SDM 2014, 2016–17
- KDD 2014–16
- IUI 2016
- RecSys 2016
- SIGMOD 2013
- DASFAA 2011