Lab Assignment 05: Classification with the Wine Dataset

Step 1. Wine dataset (15pts)

1. Load the Wine dataset using sklearn.datasets.load_wine()

2. Explore the dataset to understand the features and target classes

3. Choose the right plot to examine the classes distributions. For example, you should be able to check if classes are balanced.

4. Check if you have any missing values. If yes, apply any relevant standard approach to deal with NA.

5. Normalize dataset using StandardScaler

6. Split the dataset into training and testing sets. Suggested split ratio: 80% training, 20% testing

Step 2: Naive Bayes (NB) (20pts)

1. Import Gaussian NB from sklearn.naive_bayes import GaussianNB

   You can also consult the sklearn documentation: GaussianNB

2. Import evaluation metrics from sklearn.metrics import accuracy_score, precision_score, recall_score, ConfusionMatrixDisplay

3. Train model. As you learned from regression, sklearn has the same pipeline: fit, predict

   # Train
   model = GaussianNB()
   model.fit(X_train, y_train)   
   # Predict
   y_pred = model.predict(X_test)
   

4. Apply the evaluation metrics. These metrics will be similar for other classifiers to use.

   For precision, we use weighted average since we are dealing with multiclass classification. By default, precision_score calculates precision for each class individually. By specifying average='weighted', the metric combines these per-class precision scores into a single number.

   accuracy = accuracy_score(y_test, y_pred)
   precision = precision_score(y_test, y_pred, average='weighted')
   recall = recall_score(y_test, y_pred, average='weighted')
       
   print(f"Accuracy: {accuracy}")
   print(f"Precision: {precision}")
   print(f"Recall: {recall}")
   # Confusion matrix
   ConfusionMatrixDisplay.from_predictions(y_test, y_pred)
   plt.show()
   

Step 3: KNN (20 Points)

1. Import a KNN classifier.

2. Train a KNN classifier from sklearn.neighbors import KNeighborsClassifier

3. Assign k=3

   Note: k=3 is a commonly chosen starting point. You will later explore how to find the best k.

   
   from sklearn.model_selection import cross_val_score
   # Train model Example (no cross-validation)
   # K-Nearest Neighbors with k=3
   knn_model_k3 = KNeighborsClassifier(n_neighbors=3)
   knn_model_k3.fit(X_train, y_train)
   y_pred_knn_k3 = knn_model_k3.predict(X_test)
   
   # Evaluate KNN with k=3
   print("KNN (k=3) Evaluation:")
   print(f"Accuracy: {accuracy_score(y_test, y_pred_knn_k3):.4f}")
   print(f"Precision: {precision_score(y_test, y_pred_knn_k3, average='weighted'):.4f}")
   print(f"Recall: {recall_score(y_test, y_pred_knn_k3, average='weighted'):.4f}")
   ConfusionMatrixDisplay.from_predictions(y_test, y_pred_knn_k3)
   plt.show()
   

4. Import cross-validation from sklearn.model_selection import cross_val_score

5. Evaluate the model using K-fold cross-validation, using 5 folds

   
   scores = cross_val_score(KNeighborsClassifier(), X_train, y_train, cv=5)
   print(f"Cross-validation scores: {scores}")
   print(f"Mean score: {scores.mean()}")

6. Learn how to set a range of k and determine the best value, see Reading: Best K

7. Here is an example how you can loop over several odd k values and use cross-validation technique:

   
   #select the range, for example from 1 to 21
   k_values = range(1, 21, 2)
   # create an array to get the average metrics for each fold
   mean_scores = []
   
   for k in k_values:
       knn_model = KNeighborsClassifier(n_neighbors=k)
       scores = cross_val_score(knn_model, X_train, y_train, cv=5)
       mean_scores.append(scores.mean()) 
   

8. Find the best score (remember to import numpy) and print it:

   best_k = k_values[np.argmax(mean_scores)]
   print(f"Best k: {best_k}, Accuracy: {max(mean_scores):.4f}")
   

9. Train KNN on the best_k (see how it was done with k=3) and display the evaluation metrics

10. Bonus: Decision boundaries are typically visualized in 2D or 3D. Plotting the decision boundary with 13 features is not feasible and we learn soon how to do PCA (dimensionality reduction). To visualize plot for now, choose 2 features only, for example (you can manually choose two features from the dataset (e.g., alcohol and malic_acid)). Then you have to retrain your KNN and follow the example from the provided document to visualize the plot

Step 4: Decision Tree (20 Points)

1. Import Decision Tree from sklearn.tree import DecisionTreeClassifier

2. Train the model (see Documentation)

   Set max depth to 3: "max_depth" refers to a hyperparameter that controls the maximum depth or number of levels a tree can grow to during training

   
   # Train model
   model = DecisionTreeClassifier(max_depth=3)
   model.fit(X_train, y_train)
   

3. Import plot_tree from sklearn.tree import plot_tree

4. Plot Decision tree, using this Documentation

5. Modify the tree depth and create another plot

6. Do not forget to evaluate the model

Step 5: SVM (20 Points)

1. Import SVM from sklearn.svm import SVC

2. Train an SVM with a linear kernel

   # Train model
   model = SVC(kernel='linear')
   model.fit(X_train, y_train)

3. Experiment with other kernels (see Documentation).

4. Predict and evaluate the model for each kernel you choose

Step 6: Reflections

• What did you learn in this Lab?

• What was the most interesting (for example, specific model, evaluation etc)

• Do you feel comfortable using classifiers? Do you feel you gained useful practical experience?

Grading Rubrics

1. Step 1 Wine Dataset 10pts
2. Step 2 NB 20pts
3. Step 3 KNN 20pts
4. Step 4 Decision Tree 20pts
5. Step 5 SVM 20pts
6. Submission, Code, Format, Help citation (10 Points)