COMPSCI 646: Information Retrieval – Fall 2023

COMPSCI 646 is a graduate-level course in Information Retrieval, the science and engineering of indexing, organizing, searching, and making sense of unstructured or mostly unstructured information, particularly text. The class focuses primarily on the underlying models used for effective search and organization, but includes some discussion of efficiency concerns. The course also covers current research problems and methodologies in the field of Information Retrieval.

Tue & Thu, 4:00 – 5:15 PM

Holdsworth, Room 203

	Instructor: Hamed Zamani Contact: zamani@cs.umass.edu Office Hours: Tue, 9:00 – 10:00 AM @ CS 350
	Teaching Assistant: Yen-Chieh Lien Contact: ylien@cs.umass.edu Office Hours: Mon & Wed, 3:00–4:00 PM @ CS 207

Prerequisites

Proficiency in Python and/or Java
Basic knowledge of probability, statistics, and information theory
Foundations of applied machine learning and deep learning

Textbook

[WBC] W. Bruce Croft, Donald Metzler, and Trevor Strohman. Search Engines: Information Retrieval in Practice. Pearson Education, 2009.
[CDM] Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schütze. Introduction to Information Retrieval. Cambridge University Press, 2008.

Grading

Assignments (3×10%)
IR Talk Summaries (10%)
Midterm exam (30%): November 9, 2023 at 7 PM. Location: Herter 227.
Final project (30%)

Tentative Schedule

#	Lecture	Date	Readings
1	Introduction	Tue 9/5	[WBC] Ch.1 [WBC] Ch.7.1 [CDM] Ch.8.1, 8.2
2	IR and ML Basics	Thu 9/7
3	IR and ML Basics	Tue 9/12
4	IR Evaluation Methodologies, Metrics, and User Models	Thu 9/14	[CDM] Ch.8.3, 8.4 (optional) NDCG: Järvelin and Kekäläinen (TOIS 2002)
5	IR Evaluation Methodologies, Metrics, and User Models	Tue 9/19
6	Indexing, Vector Space Models & LSI	Thu 9/21	[WBC] Ch.4.1, 4.2, 4.3 [WBC] Ch.5.1, 5.2, 5.3, 5.4, 5.7 [CDM] Ch.6.2, 6.3, 6.4 [CDM] Ch.11.1, 11.2, 11.3, 11.4.3
7	Probabilistic Retrieval Models and BM25	Tue 9/26
8	Language Modeling	Thu 9/28	[CDM] Ch.12.1, 12.2, 12.3 Query likelihood: Ponte and Croft (SIGIR 1998) KL-divergence retrieval model: Lafferty and Zhai (SIGIR 2001) LM smoothing: Zhai and Lafferty (TOIS 2004) [WBC] Ch.11.3 [CDM] Ch.12.4 (optional) Cluster-based LM: Liu and Croft (SIGIR 2004) (optional) LDA-based LM: Wei and Croft (SIGIR 2006) (optional) Local document expansion: Tao et al. (NAACL 2006) (optional) General LM: Song and Croft (CIKM 1999) Dependence models: Metzler and Croft (SIGIR 2005) Positional LM: Lv and Zhai (SIGIR 2009)
9	Language Modeling	Tue 10/3
10	Query Expansion & Relevance Feedback	Thu 10/5	[WBC] Ch.7.3.2 Relevance models: Lavrenko and Croft (SIGIR 2001) (optional) Model-based feedback: Zhai and Lafferty (CIKM 2001) (optional) Matrix factorization model: Zamani et al. (CIKM 2016)
	Monday Schedule	Tue 10/10
11	Learning to Rank	Thu 10/12	LTR Textbook: Li (2011) Ch.1, 2, 6 (optional) LTR Textbook: Li (2011) Ch.4
12	Distributed Representation Learning for Text	Tue 10/17	word2vec: Mikolov et al. (NIPS 2013) (optional) GloVe: Pennington et al. (EMNLP 2014) (optional) Relevance-based Word Embedding: Zamani and Croft (SIGIR 2017) (optional) Embedding-based Query Expansion: Zamani and Croft (ICTIR 2016)
13	Contextual Representations and Large Language Models	Thu 10/19	(optional) ELMo: Peters et al. (NAACL 2018) Transformer: Vaswani et al. (NeurIPS 2017) BERT: Devlin et al. (NAACL 2020) BERT Re-Ranking: Nogueira and Cho (2019) Pre-trained Transformer for Ranking: Lin et al. (2022) (optional) GPT-4: OpenAI (2023)
14	Neural Ranking Models	Tue 10/24	(optional) DSSM: Huang et al. (CIKM 2013) (optional) K-NRM: Xiong et al. (SIGIR 2017) (optional) Duet Model: Mitra et al. (WWW 2017) SNRM: Zamani et al. (CIKM 2018) DPR: Karpukhin et al. (EMNLP 2020) ColBERT: Khattab and Zaharia (SIGIR 2020) (optional) ANCE: Xiong et al. (ICLR 2021)
15	Neural Ranking Models	Thu 10/26
16	Implicit Feedback, Biases, and Click Models	Tue 10/31	(optional) Implicit feedback: Joachims et al. (SIGIR 2005) (optional) Click models for web search: Chuklin et al. (2015) (optional) Counterfactual learning: Joachims and Swaminathan (SIGIR 2016) (optional) Unbiased learning to rank: Joachims et al. (WSDM 2017)
17	Link Analysis & MapReduce	Thu 11/2	[CDM] Ch.21 (optional) PageRank: Page et al. (1998) (optional) HITS: Kleinberg (JACM 1999) (optional) Waterloo spam scorer: Cormack et al. (IRJ 2011)
18	Novelty and Diversity	Tue 11/7	Maximal Marginal Relevance (MMR): Carbonell and Goldstein (SIGIR 1998) Novelty and diversity evaluation: Clarke et al. (SIGIR 2008) (optional) Subtopic retrieval using language models: Zhai et al. (SIGIR 2003) (optional) Term-level diversification: Dang and Croft (SIGIR 2013)
19	Information Filtering and Recommendation	Thu 11/9	(optional) Information Retrieval and Information Filtering: Belkin and Croft (CACM 1992) (optional) Recommender Systems Handbook: Ricci et al. (2015)
20	IR Applications: CLIR, Personalization, Entity Retrieval	Tue 11/14	(optional) CLIR: Nie et al. (2010) (optional) Web search personalization: Bennett et al. (SIGIR 2012) (optional) Expertise Retrieval: Balog (FntIR 2012)
21	IR Applications: CLIR, Personalization, Entity Retrieval	Thu 11/16
22	User Study, Crowdsourcing, and Query Log Analysis	Tue 11/21	Crowdsourcing user studies with Mechanical Turk: Kittur et al. (CHI 2008) (optional) Interactive IR evaluation: Kelly et al. (ICTIR 2015) (optional) Understanding user behavior from query logs: Dumais et al. (Ways of Knowing in HCI, 2014)
	No Class: Thanksgiving	Thu 11/23
23	Retrieval-Enhanced Machine Learning: QA, Fact Verification, and Beyond	Tue 11/28	REML: Zamani et al. (SIGIR 2022) (optional) Guided Transformer: Hashemi et al. (SIGIR 2020) (optional) FiD: Izacard and Grave (EACL 2021) (optional) RAG: Lewis et al. (NeurIPS 2020) (optional) KILT: Petroni et al. (NAACL 2021) (optional) LaMP: Salemi et al. (2023)
24		Thu 11/30
25	Context-Aware & Conversational Search	Tue 12/5	(optional) Context-sensitive language model: Shen et al. (SIGIR 2005) (optional) CIS: Zamani et al. (FntIR 2023)
26	Current IR Research	Thu 12/7

Course Policy

Late Submission

Each student has a total of 5 late days without penalty. You can use up to 3 late days per assignment, talk summary, or project milestone excluding the project’s final report. Once you use all your late submission credits, you will lose 20% (absolute) of the homework points per day.

In case of multiple submissions of an assignment, only the last one will be taken into account for the number of late days.

Collaboration and Help

You may discuss the ideas behind assignments with others. You may ask for help understanding class and IR concepts. You may study with friends. However…

The work that you submit must be your own. It may not be copied from the web, from another student in the class, or from anyone else. If you stumble upon and use a solution from the textbook or from class, you are expected to acknowledge the source of the work.

Your assignment submissions must be your own work and not in collaboration with anyone. Your project work must be your own work and not a copy of someone else’s work.

Search Google Appliance

Hamed Zamani