Natural Language Generation and Summarization

COMS 6975 — Fall 2026 — Columbia University

Subject to change

Course Overview

There has been a paradigm shift in the development of models to generate language for different purposes — classification, document summarization, generation, creative writing, and image captioning to name a few. This success has largely come about due to rapid advances in large language models such as GPT5, Claude, Qwen and many others.

In this class, we will explore four main topics: language generation, multimodal generation, summarization and ethics. We will study large language models that have been used for these tasks and the issues that arise with their use. For example, how can we control the output of these large language models along different dimensions? How do we evaluate the text generated by such systems? How can we develop models to produce or summarize creative texts? What are the ethical issues surrounding these kinds of models?

We will have some invited speakers, as shown on the syllabus. Typically an invited speaker will present for half of the class and may present remotely. Starting on Sept. 30th, half the class will consist of a debate on two papers for that day. For each of the papers, one student will present the paper from a positive point of view and a second student will critique the paper.

Time	W 4:10–6:00pm
Professor	Kathleen McKeown
Office Hours	W 1:00–2:00pm, Th 5:00–6:00pm (CEPSR 516)
Email	kathy@cs.columbia.edu

Teaching assistants and their office hours are listed below.

TA	Location	Time
Zach Horvitz (zfh2000@columbia.edu)	Schermerhorn (room TBD)	TBD
Marvin Limpijankit (ml4431@columbia.edu)	Schermerhorn (room TBD)	TBD

Requirements

Students who take the class will have four main assignments:

For each class there will be a reading assignment consisting of several research papers. Students are responsible for reading all papers.
Each student will be part of a debate in which they will be responsible for either presenting a paper or raising critiques about that paper. The class will participate in discussion of the paper following the debate. Class participation will be graded.
Each student will carry out a semester-long project. This project requires submission of: a. a proposal for the project near the beginning of class; b. a midterm progress report and c. a final report and code for their project.
Two quizzes on the class reading and debates.

Late submission policy: You have 4 free late days to use across the 3 assignments for the course (proposal, midterm, final). Once you have used all your days, then you will lose 7% of the points per day late unless you have demonstrated a valid medical excuse.

There will be no midterm or final exam.

Prerequisites

Students must have received a B or better in COMS 4705 (NLP) or equivalent. The version of NLP that you took must have covered deep learning methods for tasks in NLP.

There will be a form asking you to provide information about your background. The form also includes a short assignment required for eligibility. This form will be made available mid-August.

This is due the day after the first class (Thursday, September 10). A percentage of the class will be accepted early for forms completed before September 1. Only students who fill out the form and meet the requirements will be considered for entrance into the class — please do not request approval by email.

Syllabus

This is still being finalized. Topics, speakers, and readings may change.

Class	Date	Topic	Selected Readings
1	Sept 9	Introduction to class (Kathy McKeown) LLM basics (Zach Horvitz)	Lecture Chain-of-Thought Prompting Elicits Reasoning in Large Language Models DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning Training language models to follow instructions with human feedback
2	Sept 16	Summarization successes (Kathy McKeown) Evaluation Practice debate presentation by Zach Horvitz · Rebuttal by Marvin Limpijankit	Lecture Benchmarking Large Language Models for News Summarization ReIFE: Re-evaluating Instruction-Following Evaluation G-Eval: NLG Evaluation using Gpt-4 with Better Human Alignment Practice Debate TBD
3	Sept 23	Frontiers of summarization: narrative (Kathy McKeown) Perspective summarization (Nick Deas)	Lecture Reading Subtext: Evaluating Large Language Models on Short Story Summarization with Writers One Thousand and One Pairs: A “novel” challenge for long-context language models Modular Pluralism: Pluralistic Alignment via Multi-LLM Collaboration Reranking-based Generation for Unbiased Perspective Summarization
4	Sept 30	Beyond Next Token Prediction (Zach Horvitz) Debate presentations begin	Lecture Better & Faster Large Language Models via Multi-token Prediction Diffusion-LM Improves Controllable Text Generation Debate The Pitfalls of Next-Token Prediction Whose story is it? Personalizing story generation by inferring author styles
5	Oct 7	Frontiers in generation: persona vector generation / steering (Kathy McKeown) Debate	Lecture Persona Vectors: Monitoring and Controlling Character Traits in Language Models Debate Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs Persona Features Control Emergent Misalignment
6	Oct 14	Multimodality (Marvin Limpijankit) Debate	Lecture Cambrian-1 A Fully Open, Vision-Centric Exploration of Multimodal LLMs Vision Language Models are Biased Debate Visually Prompted Benchmarks Are Surprisingly Fragile How Do Vision-Language Models Process Conflicting Information Across Modalities?
7	Oct 21	Interpretability: overview (Kathy McKeown) Debate	Lecture Sparse Autoencoders Find Highly Interpretable Features in Language Models Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV) Debate Emotion concepts and their function in a large language model Is This Just Fantasy? Language Model Representations Reflect Human Judgments of Event Plausibility
8	Oct 28	Industry approaches Yanda Chen (Anthropic) Debate	Lecture TBD Debate Predicting LLM Safety Before Release by Simulating Deployment Tülu 3: Pushing Frontiers in Open Language Model Post-Training
9	Nov 4	Interpretability: artificial impressions (Nick Deas) Debate	Lecture Artificial Impressions: Evaluating Large Language Model Behavior Through the Lens of Trait Impressions SocioProbe: What, When, and Where Language Models Learn about Sociodemographics Debate Reading Between the Prompts: How Stereotypes Shape LLM’s Implicit Personalization Language Models as Agent Models
10	Nov 11	Ethics (Kathy McKeown) Debate	Lecture Lost in Delusion: Examining LLM Safety Under User Delusions and Distress Debate Jailbreaking Black Box Large Language Models in Twenty Queries Three Years of r/ChatGPT: Societal Impact Evaluations from Social Media Data
11	Nov 18	Agent based approaches Elias Stengel-Eskin (UT Austin) Debate	Lecture TBD Debate ReAct: Synergizing Reasoning and Acting in Language Models GAIA: a benchmark for General AI Assistants
Nov 25 — No class (Thanksgiving)
12	Dec 2	Ethics Esin Durmus (Anthropic) Debate	Lecture TBD Debate Investigating Affective Use and Emotional Well-being on ChatGPT Cognitive Reframing of Negative Thoughts through Human-Language Model Interaction
13	Dec 9	Industry views Kailash Karthik Saravana Kumar (Cohere) Class closing (Kathy)	Lecture TBD Debate Related to "The Future of AI" (exact paper TBD) Related to "The Future of AI" (exact paper TBD)

Readings

There is no textbook for the class. Readings will be assigned alongside each topic on the syllabus above as the semester is finalized.

Additional readings (optional):

Academic Integrity

Copying or paraphrasing someone's work (code included), or permitting your own work to be copied or paraphrased, even if only in part, is not allowed, and will result in an automatic grade of 0 for the entire assignment or exam in which the copying or paraphrasing was done. Your grade should reflect your own work. If you believe you are going to have trouble completing an assignment, please talk to the instructor or TA in advance of the due date.