Week 2:

• Practical: The practical in this first week of teaching will be exceptionally split into two parts:
• 2:15-3:15pm   a tutorial in CS202J (handouts are in /db/course/3/sll/slides/practical_1-six-a-side.ps)
• 3:15-4:15pm   hands-on practical in CS006 (read file README-prac01 and follow). An excerpt of the paper The Linguist's Guide to Statistics by Krenn and Samuelsson will be also used in the practical.
• Lecture 1: Introduction (slides).  Recommended reading:
• D. Kazakov. Natural  Language Processing  Applications  of Machine Learning, pp. 1-9.
• T. Mitchell. Machine Learning. pp. 20-23.
• Lecture 2:

Workload

Lectures      : 18 x 1 hr lectures
Practicals    :  8 x 2 hrs practicals
Private study : 54 hrs
Assessment    : 10 hrs

Requirements

Good practical knowledge of logic programming (Prolog) and familiarity with context-free grammars and  syntactic analysis of formal languages are expected (at the level of IPL). Students should be acquainted with the basics of natural language grammar, e.g. they should  be able to distinguish a verb from noun,  and a  past tense  from present.   NLP and  MCL  would suitably complement this module  with additional theoretical insight.  However, neither  of  those two  modules  is formally  required  as  a pre-requisite, and the overlap of material will be limited.
 

Assessment

Closed : none

Open : a report covering both theoretical and practical aspects of the subject. The discussion  will be based on the  application of existing software tools to natural language data. (50 marks)
 

Description

Machine  learning of  language studies  the acquisition  of linguistic knowledge from examples. This approach  is used in practice to develop tools  able  to  analyse  textual  data at  various  levels,  such  as morpholexical analysis, part-of-speech tagging, syntactic and semantic analysis.  The wide availability of text corpora and suitable learning techniques  make language  learning a  less costly  and  more flexible alternative to  the manual development  of tools for  natural language processing  (NLP).   While  statistical  learning methods  are  widely spread  nowadays, they  have their  clear limitations.   The  need for adequate    modelling    of     semantics,    and  acquisition  of human-comprehensible theories has led  to an increased use of symbolic learning in the recent years.

The module  will provide an insight  into some of  the most up-to-date research in machine learning, and  focus on three main issues involved in the  symbolic learning of  tailored NLP tools:

1. linguistic data and its efficient  manipulation,
2. transformation-based learning and inductive  logic  programming  as  two symbolic  learning  methods  of choice, and,
3. the application  of those methods to  particular NLP tasks (see example).

Aims

To  familiarise  with  some  of  the  most  commonly  used  linguistic resources, such  as Penn Treebank Corpus and  WordNet; become familiar with the principles of  two symbolic machine learning methods, namely, transformation-based  learning and  inductive logic  programming; gain hands-on  experience with  the  application of  these  methods to  the automatic acquisition of NLP tools.
 

Content
 

• Linguistic resources (Lectures 1-3):

Text corpora.   Penn Treebank.  Filtering Penn  Treebank with tgrep (hands-on  practicals).  Lexical databases.  The Brill Lexicon and WordNet (hands-on).
 
• Symbolic Machine Learning Methods (Lectures 4-7):

Transformation-based  learning.   Inductive  logic programming  and first-order decision lists.   Progol and CLOG (hands-on).
 
• Practical NLP (Lectures 8-10):

Morpholexical  analysis, part-of-speech  (PoS) tagging,  shallow and complete parsing, semantic disambiguation.  Evaluation criteria.
 
• Symbolic learning of language (Lectures 11-18):

The  Brill Part-of-Speech  Tagger.  Predicting  the PoS  of unknown words with the Brill  tagger (hands-on).  Learning contextual rules  for PoS tagging (hands-on).  Prediction of the PoS of unknown words with the ILP systems  Progol and CLOG (hands-on). Learning tailored parsers with  the ILP  system LAPIS; use  of lexical  semantics for syntactic disambiguation (hands-on).

Lecture  slides and  important papers  will be  made available  on the module WWW page.
 

Recommended books

** Brill E. A Corpus-Based  Approach to Language Learning. PhD Thesis,
   UPenn, 1993. URL: http://www.cs.jhu.edu/~brill/dissertation.ps

** Kazakov  D.  Natural  Language Processing  Applications  of Machine
   Learning.  PhD Thesis,  Czech Technical  University,  Prague, 1999.
   URL: ftp://ftp.cs.york.ac.uk/pub/aig/Papers/dimitar.kazakov/content.ps.gz

** Roberts S. An Introduction  to Progol. Technical Manual. University
   of York, 1997. URL: ftp://ftp.cs.york.ac.uk/pub/ML_GROUP/Papers/progol-intro.ps.gz

++ Ritchie  G.,  Russell G.,  Black  A.   and  Pulman, S.   Computational
   Morphology:  Practical Mechanisms  for the  English Lexicon.  The MIT
   Press, 1992.

++ Pereira F., Shieber S. Prolog and Natural Language Analysis. CSLI/SRI,
   1987.

+  Mitchell T. Machine Learning. McGraw-Hill, 1997.

+ Antal van den Bosch. Basics of memory-based and decision-tree learning (On-line visual demo).