Classification Exercises
Exercise 1.
1.
Fire up the Weka (Waikato Environment for Knowledge Analysis) soft-
ware,
launch the explorer window and select the \Preprocess" tab. Open the iris
data-set (\iris.ar_", this should be in the ./data/ directory of the Weka
install).
2.
Select the \Classify" tab. Under the \Test options" section you have four
different testing options. How do each (we cannot use \supplied test set"
option as we have no applicable _le) of these options select the
training/testing? Which testing mode do you think will perform best? (the
ExplorerGuide.pdf", in the ./
directory of the Weka install may help).
3.
Under \Classifier" select \MultilayerPerceptron". What type of
classifier is this? How does this classifier work? What main parameters can be
specified for this classifier?
4.
Under \Test options" select \Use training set" and under \More options"
check \Output predictions". Now click \Start" to start training the
model. You should see a stream of output appear in the window named \Classifier
output". What do each of the following sections tell you about the model?
(a)
\Predictions on ..."
(b)
\Summary"
(c)
\Detailed accuracy by class"
(d)
\Confusion matrix"
5.
Under \Results list" you should see your model, right click on it and select
\Visualise classifier errors", points marked with a square are errors i.e.
incorrectly classified. How do you think the classifier performed on the test
data?
6.
Under \Test options" vary the option selected i.e. \cross-validation"
or \percentage" and their parameters i.e. \folds" and \%". Then
start the training phase again for each model. For each model analyse the classifier
output and visualise the classifier errors. How do the different training
techniques affect the model? Which technique performed the
best?
How does this compare to your initial prediction in 4?
7.
Repeat the exercise 6 with the \J48" (Decision Tree) and \RBFNetwork"
classifiers. How do these compare to each other? How do these compare to the
MultilayerPerceptron"?