Machine Learning Basics with Scikit-learn

Objective

To introduce students to the fundamental concepts and techniques of machine learning using the Scikit-learn library.

Prerequisites

For convenience you can use the terminal window at the OReilly interactive lab: https://learning.oreilly.com/scenarios/ethical-hacking-advanced/9780137673469X002/

1. Basic understanding of Python programming.
2. Familiarity with data manipulation libraries like Pandas and NumPy.
3. Python and necessary libraries installed: Scikit-learn, Pandas, and NumPy.

Lab Outline

1. Introduction to Machine Learning:

   • Brief explanation of machine learning and its types (Supervised, Unsupervised).
   • Introduction to Scikit-learn library.

2. Setting Up the Environment:

   • Installing Scikit-learn, Pandas, and NumPy:

     pip3 install scikit-learn pandas numpy
     

3. Data Preprocessing:

4. Step 1: Importing Necessary Libraries:

   import numpy as np
   import pandas as pd
   from sklearn import datasets
   

5. Step 2: Loading a Dataset:

   iris = datasets.load_iris()
   X, y = iris.data, iris.target
   

6. Step 3: Handling Missing Values (if any):

   # Using SimpleImputer to fill missing values
   from sklearn.impute import SimpleImputer
   imputer = SimpleImputer(strategy="mean")
   X_imputed = imputer.fit_transform(X)
   

7. Step 4: Splitting the Dataset into Training and Testing Sets:

   from sklearn.model_selection import train_test_split
   X_train, X_test, y_train, y_test = train_test_split(X_imputed, y, test_size=0.2, random_state=42)
   

8. Building Machine Learning Models:

9. Step 5: Training a Decision Tree Model:

   from sklearn.tree import DecisionTreeClassifier
   dt_classifier = DecisionTreeClassifier(random_state=42)
   dt_classifier.fit(X_train, y_train)
   

10. Step 6: Training a Logistic Regression Model:

    from sklearn.linear_model import LogisticRegression
    lr_classifier = LogisticRegression(random_state=42)
    lr_classifier.fit(X_train, y_train)
    

11. Evaluating Models:

12. Step 7: Making Predictions and Evaluating Models:

    from sklearn.metrics import accuracy_score
    
    # For Decision Tree
    y_pred_dt = dt_classifier.predict(X_test)
    dt_accuracy = accuracy_score(y_test, y_pred_dt)
    
    # For Logistic Regression
    y_pred_lr = lr_classifier.predict(X_test)
    lr_accuracy = accuracy_score(y_test, y_pred_lr)
    
    print(f"Decision Tree Accuracy: {dt_accuracy}")
    print(f"Logistic Regression Accuracy: {lr_accuracy}")
    

13. Hyperparameter Tuning and Cross-Validation:

14. Step 8: Implementing Grid Search Cross-Validation:

    from sklearn.model_selection import GridSearchCV
    
    # For Decision Tree
    param_grid_dt = {'max_depth': [3, 5, 7], 'min_samples_split': [2, 5, 10]}
    grid_search_dt = GridSearchCV(dt_classifier, param_grid_dt, cv=3)
    grid_search_dt.fit(X_train, y_train)
    
    # Best parameters and score for Decision Tree
    print(grid_search_dt.best_params_)
    print(grid_search_dt.best_score_)
    

15. Conclusion and Further Exploration:

16. Discuss the results and explore how to further improve the models.

17. Introduce more advanced machine learning techniques and algorithms.

18. Assignment/Project:

19. Assign a project where students have to apply the techniques learned in the lab to a real-world dataset and build a predictive model.

Assessment

• Lab Participation: Active participation in lab exercises.
• Quiz: Conduct a short quiz to assess the understanding of students regarding the concepts taught in the lab.
• Project Evaluation: Evaluate the project based on the application of concepts, the accuracy of the model, and the presentation of results.

Resources

1. Scikit-learn documentation for detailed guidance on using the library.
2. Online courses and tutorials to further explore machine learning concepts.

By the end of this lab, students should be able to understand and implement basic machine learning concepts using the Scikit-learn library. They should also be capable of building and evaluating simple machine learning models.