Statistical methods in computational linguistics

Spring 2005
MW 7:00–8:15
Room BA-412

This course offers an introduction to statistical methods in computational linguistics. Through a combination of lectures, demonstrations, and hands-on exercises, this course will give students an introduction to the skills necessary for evaluating constructing statistical natural language processing applications and for evaluating their results. Topics to be covered include:

• basic probability and information theory
• statistics for corpus analysis and hypothesis testing
• Markov chains and sequence models
• probabilistic context-free grammars
• stochastic attribute value grammars

Pre-/co-requisite: Ling 581 or equivalent (some experience with python or a similar scripting language will be helpful)

Instructor

Rob Malouf
Office: BA 310A
Office Hours: Mondays 1:30-3:00, or by appointment
Email: rmalouf@mail.sdsu.edu
Phone: (619) 594-7111

Requirements

The final grade will be based on homework assignments (20%), a take-home midterm exam (30%), and a final project (50%).

Through the term, there will be occasional homework assignments to practice the techniques learned in class. Working in groups is encouraged, but please include the names of all coworkers on the assignment.

The final project for this course will be a group project to design, implement, document, and evaluate an NLP application based on the statistical methods covered in the course. The details will depend on the interests of the students, but one possible project would be a system which can correctly answer `fill-in-the-blank' vocabulary questions from exams like the SAT or GRE.

Readings

The required textbooks for this course are:

Christopher D. Manning and Hinrich Schütze. 1999. Foundations of Statistical Natural Language Processing. MIT Press.

Eugene Charniak. 1993. Statistical Language Learning. Cambridge: MIT Press.

They are for sale in the campus bookstore and at Amazon, etc. Updates and corrections to the first book can be downloaded from the authors' website.

Additional readings will be made available in class or via the "Resources" section of the course web page.

Schedule

Week 1–4    Introduction (Chapters 1, 2)
Background · Mathematical background · Probability · Information Theory

Week 5–8 Statistics (Chapters 3, 4, 5)
Descriptive statistics · Graphical methods · Hypothesis testing · Paired sign test · Bootstrap · Corpus statistics

Week 9–10 Sequence models (Chapters 6, 9, 10)
N grams · Smoothing · Hidden Markov models · Viterbi decoding · Part of speech tagging

Week 11–14 Parsing (Chapters 11, 12)
Probabilistic context free grammars · Inside-Outside algorithm · Treebank grammars · Dependency-based models

Week 15 Class projects

Final project
Project due May 18

Resources

• Links
  
  • Galton's quincunx
  • The R Project for Statistical Computing
  • John Verzani's simpleR tutorial
  • ANC frequency data
  • Files for midterm:
    baseline.py, baseline.pl, train.txt, test.txt
  • Chunking shared task
    train.txt.gz, test.txt.gz, conlleval
• Slides
  
  • Introduction
    01-intro.pdf (print), 02-intro.pdf (print)
  • Probability
    03-prob.pdf (print), 04-prob.pdf (print)
  • Information Theory
    05-info.pdf (print), 06-info.pdf (print)
  • Statistics
    08-stat.pdf (print), 09-stat.pdf (print), 10-stat.pdf (print), 11-stat.pdf (print), 12-stat.pdf (print)
  • Sequence models
    13-seq.pdf (print), 14-seq.pdf (print), 15-seq.pdf (print), 16-seq.pdf (print), 17-seq.pdf (print), 18-seq.pdf (print)
  • Hidden Markov Models
    19-seq.pdf (print), 20-seq.pdf (print), 21-seq.pdf (print)
  • Parsing models
    22-seq.pdf (print), 23-seq.pdf (print), 24-seq.pdf (print)
  • Machine learning
    25-ml.pdf (print), 26-ml.pdf (print), 28-ml.pdf (print), 29-ml.pdf (print), 30-ml.pdf (print)

rmalouf@mail.sdsu.edu
Last modified: Tue May 24 13:14:43 PDT 2005