CSI 7110, Spring 2025
Topics in Responsible AI

Department of Computer Science and Engineering & Department of Artificial Intelligence
Yonsei University

Fridays at 10:00 AM - 1:00 PM
D408

Prof. Minsuk Kahng
Assistant Professor, Department of Computer Science and Engineering
E-mail: [email protected]
Website: https://minsuk.com

Course Description

As artificial intelligence (AI) continues to transform society, recognizing the importance of Responsible AI has become essential. Responsible AI refers to the development and deployment of AI systems with an awareness of their broader societal impact, considering principles such as fairness, accountability, transparency, interpretability, and safety. This course explores cutting-edge research on Responsible AI from academia and industry, emphasizing a human-centered perspective. In particular, we will examine the role of data in the AI development lifecycle, as well as the role of people who design and interact with these AI systems. Through this course, students will develop knowledge, tools, and skills necessary to design AI systems that are not only accurate but also equitable, transparent, and trustworthy.

Course Topics

This course will cover the following topics (subject to change).

Role of Data
Fairness
Evaluation
Explainability
Safety
Visualization
Labeling and Participatory AI

Course Schedule Tentative

This is a tentative schedule. For up-to-date information, please check out LearnUs.

Week	Topic	Readings
1	Course Introduction
2	Role of Data	On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜 (Emily Bender et al., FAccT 2021) Advances, challenges and opportunities in creating data for trustworthy AI (Weixin Liang et al., Nature Machine Intelligence, 2022)
3	Fairness (1)	Machine Bias (Julia Angwin et al., ProPublica, 2016) Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings (Tolga Bolukbasi et al., NIPS 2016)
4	Fairness (2)	Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification (Joy Buolamwini and Timnit Gebru, FAccT 2018) Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need? (Kenneth Holstein et al., CHI 2019)
5	Evaluation (1): Generative AI	Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena (Lianmin Zheng et al., NeurIPS 2023 D&B) Hands-On Activity
6	Evaluation (2): Behavioral Testing	Beyond Accuracy: Behavioral Testing of NLP Models with CheckList (Marco T. Ribeiro et al., ACL 2020) Adaptive Testing and Debugging of NLP Models (Marco T. Ribeiro and Scott Lundberg, ACL 2022)
7	Explainability (1)	[Guest Lecture] Sunnie S. Y. Kim: Advancing Responsible AI with Human-Centered Evaluation "Why Should I Trust You?": Explaining the Predictions of Any Classifier (Marco T. Ribeiro et al., KDD 2016)
8	Explainability (2)	Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV) (Been Kim et al., ICML 2018) Estimating Training Data Influence by Tracing Gradient Descent (Garima Pruthi et al., NeurIPS 2020)
9	Project Proposals
10	Safety	Red Teaming Language Models with Language Models (Ethan Perez et al., EMNLP 2022) Adversarial Nibbler: An Open Red-Teaming Method for Identifying Diverse Harms in Text-to-Image Generation (Jessica Quaye et al., FAccT 2024)
11	Project Posters
12	Visualization	[Lecture] Visualization for Interpretability Exploring Empty Spaces: Human-in-the-Loop Data Augmentation (Catherine Yeh et al., CHI 2025)
13	Labeling and Participatory AI	Revolt: Collaborative Crowdsourcing for Labeling Machine Learning Datasets (Joseph Chee Chang et al., CHI 2017) Jury Learning: Integrating Dissenting Voices into Machine Learning Models (Mitchell Gordon et al., CHI 2022)
14	Project Presentations
15	Project Presentations

Course Structure

This course consists of paper presentations, discussions, in-class activities, and a project. Each week, students will read 1–2 papers in advance, and a designated student will present one of them during class. We will cover papers from major AI conferences as well as related venues such as CHI and FAccT. After the presentation, we will engage in a role-playing style discussion. Towards the end of the semester, students will work on individual or team projects. There will be no exams. Weekly topics and the course format may be adjusted based on factors such as enrollment numbers.

Role-Playing Style Discussions

We adapt the role-playing paper reading seminar format described by Alec Jacobson and Colin Raffle at https://colinraffel.com/blog/role-playing-seminar.html. For each paper, students will be assigned one of the specific roles, adapted for this course.

Paper Reviewer. The paper has not been published yet and is currently submitted to a top conference where you've been assigned as a peer reviewer. Complete a full review of the paper by indicating strengths, weaknesses, and/or constructive feedback. This includes recommending whether to accept or reject the paper. You don't need to include a paper summary.
Academic Researcher. You're a researcher who is working on a new project in this area. Propose an imaginary follow-up project not just based on the current but only possible due to the existence and success of the current paper.
Industry Practitioner. You work at a company or organization developing an application or product of your choice. Bring a convincing pitch for why you should be paid to apply the method in the paper, and discuss at least one positive and/or negative impact of this application.
Policymaker. You are a policymaker shaping regulations that address the opportunities and risks of AI. Assess the societal risks discussed in this paper and propose policy recommendations. What can you do for the public sector or certain communities? How should policies navigate trade-offs between technological advancements and societal concerns?
Archaeologist. This paper was found buried under ground in the desert. You're an archeologist who must determine where this paper sits in the context of previous and subsequent work. Find and report on one older paper cited within the current paper that substantially influenced the current paper and/or one newer paper that cites this current paper.
Private Investigator. You are a detective who needs to run a background check on one of the paper's authors. Where have they worked? What did they study? What previous projects might have led to working on this one? What motivated them to work on this project?

Each paper session will follow this structure: