Foundational
Course: Probabilistic Models for NLP
Lecture 1:
Motivation for Probablistic Models for NLP and Probability Theory

Lecture 2:
Statistics, MaximumLikelihood, Learning and nextword prediction
 Lecture
Slides (for lectures 13)
 Chapter 6 from Jurafsky and Martin
and
 Sections 6.16.3 + 9.1 from
Manning and Schutze boek
 Chapter 7 from Tom Mitchell
(Machine Learning) about Bayesian Learning: recommended

Lecture 3:
POStagging Using Markov Models, Hidden Markov Models
 Lecture
Slides (for lectures 13)
 Read
chapter 8 (Jurafsky and Martin) about POS tagging in general (you
may skip section 8.6)
 On
HMMs: read from chapter 9 (Manning and Schutze) sections 9.1+9.2
+9.3.1+9.3.2

Further on evaluation of Taggers: read section 10.6
(Manning and Schutze)

Lecture 4:
Probabilistic Modeling of ContextFree Grammars
 Read
the EM
tutorial's section 5, subsection "Background: Probabilistic Modeling
of CFGs"
 Chapter
9 of the book of Jurafsky and Martin recommended

Lecture 5:
Resolving Ambiguities with Probabilistic ContextFree Grammars
 Read
the EM
tutorial's section 5, subsection "Background: Resolving Ambiguities
with Probabilistic CFGs"
 Chapter
12 of the book of Jurafsky and Martin
recommended

Chapters 5 and 8 of Charniak's book recommended


Advanced
Course: Probabilistic Parsing
Lecture 1: General
Estimation Methods and a CFG's Probability Model
 Read the EM
tutorial's section 2 "Estimation Methods" (without subsection
"MaximumEntropy Estimation")
 Read the EM
tutorial's section 5, subsection "MaximumLikelihood Estimation
of PCFGs" (the first two theorems only)

Lecture 2: MaximumLikelihood
Estimation of PCFGs and an Excursus to MaximumEntropy Modeling
 Read the EM
tutorial's section 5, subsection "MaximumLikelihood Estimation
of PCFGs" (the central theorem only: treebank grammars are maximumlikelihhod
estimates)
 Read the EM
tutorial's section 2, subsection "MaximumEntropy Estimation"

Lecture 3: EM Training of PCFGs
 Read
the EM
tutorial's section 3 "The ExpectationMaximization Algorithm"
 Read the EM
tutorial's section 5, subsection "EM Training of PCFGs"

Chapters 5 and 8 of Charniak's book recommended

Lecture 4:
Transforms on PhraseStructure Treebank for More Accurate PCFG Parsing
 Lecture
Slides (a brief coverage of these models with a unifying view)
 Michael Collins
(1997), Proceedings of ACL'97
 Eugene
Charniak(1996). Statistical parsing with a contextfree grammar and word
statistics, Proceedings of the Fourteenth National Conference on
AAAI Press/MIT Press, Menlo Park (1997). An abstract
and postscript version
 Eugene Charniak(1996). Treebank grammars,
Technical Report CS9602, Department of Computer Science, Brown University
(1996). An abstract
and postscript
version
 Eisner, Jason
M. 1996. Three new probabilistic
models for dependency parsing: An exploration. Proceedings of COLING96,
Copenhagen, August.
 M. Johnson (1999) ``PCFG models of linguistic
tree representations'' Computational Linguistics,
available in Gzipped Postscript
format or Adobe PDF format

Lecture 5:
DataOriented Parsing

