
.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_examples/model_selection/plot_confusion_matrix.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_auto_examples_model_selection_plot_confusion_matrix.py>`
        to download the full example code or to run this example in your browser via JupyterLite or Binder.

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_examples_model_selection_plot_confusion_matrix.py:


==============================================================
Evaluate the performance of a classifier with Confusion Matrix
==============================================================

Example of confusion matrix usage to evaluate the quality
of the output of a classifier on the iris data set. The
diagonal elements represent the number of points for which
the predicted label is equal to the true label, while
off-diagonal elements are those that are mislabeled by the
classifier. The higher the diagonal values of the confusion
matrix the better, indicating many correct predictions.

The figures show the confusion matrix with and without
normalization by class support size (number of elements
in each class). This kind of normalization can be
interesting in case of class imbalance to have a more
visual interpretation of which class is being misclassified.

Here the results are not as good as they could be as our
choice for the regularization parameter C was not the best.
In real life applications this parameter is usually chosen
using :ref:`grid_search`.

.. GENERATED FROM PYTHON SOURCE LINES 26-73

.. code-block:: Python


    # Authors: The scikit-learn developers
    # SPDX-License-Identifier: BSD-3-Clause

    import matplotlib.pyplot as plt
    import numpy as np

    from sklearn import datasets, svm
    from sklearn.metrics import ConfusionMatrixDisplay
    from sklearn.model_selection import train_test_split

    # import some data to play with
    iris = datasets.load_iris()
    X = iris.data
    y = iris.target
    class_names = iris.target_names

    # Split the data into a training set and a test set
    X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0)

    # Run classifier, using a model that is too regularized (C too low) to see
    # the impact on the results
    classifier = svm.SVC(kernel="linear", C=0.01).fit(X_train, y_train)

    np.set_printoptions(precision=2)

    # Plot non-normalized confusion matrix
    titles_options = [
        ("Confusion matrix, without normalization", None),
        ("Normalized confusion matrix", "true"),
    ]
    for title, normalize in titles_options:
        disp = ConfusionMatrixDisplay.from_estimator(
            classifier,
            X_test,
            y_test,
            display_labels=class_names,
            cmap=plt.cm.Blues,
            normalize=normalize,
        )
        disp.ax_.set_title(title)

        print(title)
        print(disp.confusion_matrix)

    plt.show()




.. rst-class:: sphx-glr-horizontal


    *

      .. image-sg:: /auto_examples/model_selection/images/sphx_glr_plot_confusion_matrix_001.png
         :alt: Confusion matrix, without normalization
         :srcset: /auto_examples/model_selection/images/sphx_glr_plot_confusion_matrix_001.png
         :class: sphx-glr-multi-img

    *

      .. image-sg:: /auto_examples/model_selection/images/sphx_glr_plot_confusion_matrix_002.png
         :alt: Normalized confusion matrix
         :srcset: /auto_examples/model_selection/images/sphx_glr_plot_confusion_matrix_002.png
         :class: sphx-glr-multi-img


.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Confusion matrix, without normalization
    [[13  0  0]
     [ 0 10  6]
     [ 0  0  9]]
    Normalized confusion matrix
    [[1.   0.   0.  ]
     [0.   0.62 0.38]
     [0.   0.   1.  ]]




.. GENERATED FROM PYTHON SOURCE LINES 74-87

Binary Classification
=====================

For binary problems, :func:`sklearn.metrics.confusion_matrix` has the `ravel` method
we can use get counts of true negatives, false positives, false negatives and
true positives.

To obtain true negatives, false positives, false negatives and true
positives counts at different thresholds, one can use
:func:`sklearn.metrics.confusion_matrix_at_thresholds`.
This is fundamental for binary classification
metrics like :func:`~sklearn.metrics.roc_auc_score` and
:func:`~sklearn.metrics.det_curve`.

.. GENERATED FROM PYTHON SOURCE LINES 87-125

.. code-block:: Python


    from sklearn.datasets import make_classification
    from sklearn.metrics import confusion_matrix_at_thresholds

    X, y = make_classification(
        n_samples=100,
        n_features=20,
        n_informative=20,
        n_redundant=0,
        n_classes=2,
        random_state=42,
    )

    X_train, X_test, y_train, y_test = train_test_split(
        X, y, test_size=0.3, random_state=42
    )

    classifier = svm.SVC(kernel="linear", C=0.01, probability=True)
    classifier.fit(X_train, y_train)

    y_score = classifier.predict_proba(X_test)[:, 1]

    tns, fps, fns, tps, threshold = confusion_matrix_at_thresholds(y_test, y_score)

    # Plot TNs, FPs, FNs and TPs vs Thresholds
    plt.figure(figsize=(10, 6))

    plt.plot(threshold, tns, label="True Negatives (TNs)")
    plt.plot(threshold, fps, label="False Positives (FPs)")
    plt.plot(threshold, fns, label="False Negatives (FNs)")
    plt.plot(threshold, tps, label="True Positives (TPs)")
    plt.xlabel("Thresholds")
    plt.ylabel("Count")
    plt.title("TNs, FPs, FNs and TPs vs Thresholds")
    plt.legend()
    plt.grid()

    plt.show()



.. image-sg:: /auto_examples/model_selection/images/sphx_glr_plot_confusion_matrix_003.png
   :alt: TNs, FPs, FNs and TPs vs Thresholds
   :srcset: /auto_examples/model_selection/images/sphx_glr_plot_confusion_matrix_003.png
   :class: sphx-glr-single-img






.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 0.242 seconds)


.. _sphx_glr_download_auto_examples_model_selection_plot_confusion_matrix.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: binder-badge

      .. image:: images/binder_badge_logo.svg
        :target: https://mybinder.org/v2/gh/scikit-learn/scikit-learn/1.8.X?urlpath=lab/tree/notebooks/auto_examples/model_selection/plot_confusion_matrix.ipynb
        :alt: Launch binder
        :width: 150 px

    .. container:: lite-badge

      .. image:: images/jupyterlite_badge_logo.svg
        :target: ../../lite/lab/index.html?path=auto_examples/model_selection/plot_confusion_matrix.ipynb
        :alt: Launch JupyterLite
        :width: 150 px

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_confusion_matrix.ipynb <plot_confusion_matrix.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_confusion_matrix.py <plot_confusion_matrix.py>`

    .. container:: sphx-glr-download sphx-glr-download-zip

      :download:`Download zipped: plot_confusion_matrix.zip <plot_confusion_matrix.zip>`


.. include:: plot_confusion_matrix.recommendations


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_
