Introduction to Probability and Information Theory

LSA Summer Institute 2011

Rob Malouf
Dept. of Linguistics and Asian/Middle Eastern Languages
San Diego State University

Web: http://rohan.sdsu.edu/~malouf
Email: rmalouf@mail.sdsu.edu
Institute Office Hours: IBS Cubicle 12, Mon 2:00–3:00

Linguistics as field is in the process of undergoing a dramatic methodological shift. Quantitative advances in allied fields (such as psycholinguistics, sociolinguistics, and computational linguistics) have led to a new interest in empirical methods among theoretical linguists. In order to understand and contribute to current research in linguistics, practitioners need a solid grounding in probability, statistics and information theory, areas which get little attention in traditional (under)graduate linguistics education. This course aims to fill that gap, giving students some of the mathematical background they need to be a linguist in the 21st century.

Readings

Syllabus

  1. Probability and counting [slides]
  2. Conditional probability, Bayes' Theorem [slides]
  3. Probability distributions [slides]
  4. Statistics and hypothesis testing [slides]
  5. Word frequencies [slides]
  6. Information theory: entropy and compression [slides]
  7. Information theory: noisy channels and complexity [slides]
  8. Applications to morphology [slides]

Assignments

  1. Read Grinstead and Snell, section 1.2, 3.1, 4.1;
    Do in G & S: p. 35 #4, 5, 6; p. 88 #3, 8, 16; p. 150 #1, 2, 9, 18
    [solution]
  2. Read Grinstead and Snell, section 5.1, 6.1;
    Read Verzani, pp. 1–18;
    Do in G & S: p. 154 #30, 46, 56; p. 197 #1, 2; p. 247 #1, 3
    [solution]
  3. Read Grinstead and Snell, section 8.1;
    Read Baayen, ch. 1;
    Do in G & S: p. 197 #7, 14, 20, 24; p. 247 #12, 19, 25
    [solution]
  4. Read Verzani, pp. 19–68;
    Do in Verzani: p. 47 #6.3, 6.7, 6.8; p. 53 #7.8; p. 65 #9.1, 9.2
    [solution]
  5. Read Pierce, ch. 2
    Do in G & S: p. 35 #1, 7; p. 88 #6, 13; p. 150 #4, 12; p. 197 #8, 16
    Turn in on Monday in class or submit using culearn
    [solution]
  6. Read Pierce, ch. 3, 4, 5
    Do this problem set
    [solution]
  7. Read Pierce, ch. 6, 7, 8
    Read Baayen (2003)
    Do this problem set
    Turn in on Monday in class or submit using culearn



rmalouf@mail.sdsu.edu
Last modified: Mon Aug 1 10:34:12 PDT 2011
Disclaimer