GSLT ILP Practical 1

Warming up

In this first exercise you are not required to solve a problem, the point is to get you familiar with Aleph. Make yourself a directory (i.e. a folder) to do your ILP work in. Now download Aleph and save it in your ILP directory. It's just one big Prolog program that works with both YAP prolog and SWI prolog. You will use SWI Prolog.

Now grab these 7 Aleph input files to play with: m.fold.b, m.fold.f, m.fold.n, rest.f and rest.n. Save them all to your ILP directory.

Next start up SWI Prolog. On my system I do this by typing pl at the UNIX prompt. Once you have successfully started SWI Prolog, type [aleph]. at the Prolog prompt. This consults (i.e. reads in) the file aleph.pl.

Now use the Aleph-defined predicate read_all/1 to read in background knowledge, positive and negative examples for this Slovene disambiguation problem. Do this by typing read_all('m.fold'). at the Prolog prompt. Next ask Aleph to induce rules from the data by typing induce. at the Prolog prompt.

After a minute or so, the Aleph run should finish and output results tables and accuracy for the training data and also for the test examples, which are in the files rest.f and rest.n. Note that the test set accuracy is considerably worse than the training set accuracy. This is an example of overfitting: the rules are, to a large extent picking, up chance regularities in the training examples.

You will find that the Aleph output scrolls by too quickly for you to see what was going on. However, a lot of information about the run will have been saved in a log file called m.fold.lhslog in your ILP directory. Have a look at it. Note that it starts with a record of the parameter values that were used for the run Aleph just did. Then there is a record of the search: an example is saturated, a number of clauses are constructed in the search for the best clause and the best clause found is recorded. Note that often the 'best clause' is just the example itself. This happens when Aleph fails to find an acceptable rule covering the example. At the end of the log file there is a list of the rules found together with the examples each covers. Finally the train and test results are given.

Now here's some alterations to try:

  1. Currently, the noise parameter is not set in m.fold.b so it takes its default value of 0. (You can check this by looking at the log file m.fold.lhslog.) This means that no rule can cover any negative examples. Change this by adding the line:
    :- set(noise,9999).
    to the m.fold.b file. For neatness, put it just above the line which sets minacc. Now rules can cover 9999 negative examples! However minacc is set to 0.97 meaning that each rule must have 97% accuracy on the training set. Reduce this value to 0.80 and save the altered file m.fold.b. To be on the safe side, kill your SWI Prolog session and restart a new one. Induce some rules by taking the same steps as before. Note that despite making the noise and minacc parameters more lenient, both training and test accuracies go up. This is because our new rules cover many more positives. Note that the new results of the new run are appended to the log file.
  2. Force Aleph to do pure hill-climbing search by setting the openlist parameter to 1. Normally this would lead to a massive speed-up. Does it here?
  3. Comment out your openlist parameter setting. Now alter m.fold.b so that the right context of the focus word is taken into account. Do this by uncommenting all of the modeb declarations which refer to the type msdlist except for those concerning punctuation. msdlist is the type for words in the right context. Do a run. What happens when you turn hill-climbing back on by uncommenting your openlist parameter setting and doing another run?
  4. Finally, try saturating a few individual examples. For example number 1, you can do this by typing sat(1). at the prompt instead of induce.. Try and understand what is going on by commenting out modeb declarations and seeing the effect it has.

Getting the modes right

Grab the following three files: sat_example.b sat_example.f and sat_example.n. Examine the files and check you understand them. Add the correct mode declarations to sat_example.b, so that when you saturate the first example by doing sat(1). you get the following bottom clause:

vp(A, B) :-
   vp(A, C), word(ran, A, C), modif(C, B), word(quickly, C, B).

Don't give up too soon, but the answer is available.

Inducing a single clause theory

Continuing with the sat_example example, it is clear that with this data one possible target theory is the single rule which says that a vp is either a vp followed by a modif or a single vp. Alter your background knowledge, so that when you type induce. Aleph induces a single clause representing this rule which covers both positive examples. Tip: this is easier to do if Aleph choose the first positive example to saturate, so ensure that this happens by adding :- set(samplesize,0). to your background file. This ensures that examples are saturated in order.

Here's how I did it.

Last modified: Mon Oct 10 09:47:36 BST 2005

Valid XHTML 1.1!