
.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_examples/linear_model/plot_logistic_l1_l2_sparsity.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_auto_examples_linear_model_plot_logistic_l1_l2_sparsity.py>`
        to download the full example code or to run this example in your browser via JupyterLite or Binder.

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_examples_linear_model_plot_logistic_l1_l2_sparsity.py:


==============================================
L1 Penalty and Sparsity in Logistic Regression
==============================================

Comparison of the sparsity (percentage of zero coefficients) of solutions when
L1, L2 and Elastic-Net penalty are used for different values of C. We can see
that large values of C give more freedom to the model.  Conversely, smaller
values of C constrain the model more. In the L1 penalty case, this leads to
sparser solutions. As expected, the Elastic-Net penalty sparsity is between
that of L1 and L2.

We classify 8x8 images of digits into two classes: 0-4 against 5-9.
The visualization shows coefficients of the models for varying C.

.. GENERATED FROM PYTHON SOURCE LINES 17-87



.. image-sg:: /auto_examples/linear_model/images/sphx_glr_plot_logistic_l1_l2_sparsity_001.png
   :alt: L1 penalty, Elastic-Net l1_ratio = 0.5, L2 penalty
   :srcset: /auto_examples/linear_model/images/sphx_glr_plot_logistic_l1_l2_sparsity_001.png
   :class: sphx-glr-single-img


.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    C=1.00
    Sparsity with L1 penalty:                4.69%
    Sparsity with Elastic-Net penalty:       4.69%
    Sparsity with L2 penalty:                4.69%
    Score with L1 penalty:                   0.90
    Score with Elastic-Net penalty:          0.90
    Score with L2 penalty:                   0.90
    C=0.10
    Sparsity with L1 penalty:                29.69%
    Sparsity with Elastic-Net penalty:       14.06%
    Sparsity with L2 penalty:                4.69%
    Score with L1 penalty:                   0.90
    Score with Elastic-Net penalty:          0.90
    Score with L2 penalty:                   0.90
    C=0.01
    Sparsity with L1 penalty:                84.38%
    Sparsity with Elastic-Net penalty:       68.75%
    Sparsity with L2 penalty:                4.69%
    Score with L1 penalty:                   0.86
    Score with Elastic-Net penalty:          0.88
    Score with L2 penalty:                   0.89






|

.. code-block:: Python


    # Authors: The scikit-learn developers
    # SPDX-License-Identifier: BSD-3-Clause

    import matplotlib.pyplot as plt
    import numpy as np

    from sklearn import datasets
    from sklearn.linear_model import LogisticRegression
    from sklearn.preprocessing import StandardScaler

    X, y = datasets.load_digits(return_X_y=True)

    X = StandardScaler().fit_transform(X)

    # classify small against large digits
    y = (y > 4).astype(int)

    l1_ratio = 0.5  # L1 weight in the Elastic-Net regularization

    fig, axes = plt.subplots(3, 3)

    # Set regularization parameter
    for i, (C, axes_row) in enumerate(zip((1, 0.1, 0.01), axes)):
        # Increase tolerance for short training time
        clf_l1_LR = LogisticRegression(C=C, l1_ratio=1, tol=0.01, solver="saga")
        clf_l2_LR = LogisticRegression(C=C, l1_ratio=0, tol=0.01, solver="saga")
        clf_en_LR = LogisticRegression(C=C, l1_ratio=l1_ratio, tol=0.01, solver="saga")
        clf_l1_LR.fit(X, y)
        clf_l2_LR.fit(X, y)
        clf_en_LR.fit(X, y)

        coef_l1_LR = clf_l1_LR.coef_.ravel()
        coef_l2_LR = clf_l2_LR.coef_.ravel()
        coef_en_LR = clf_en_LR.coef_.ravel()

        # coef_l1_LR contains zeros due to the
        # L1 sparsity inducing norm

        sparsity_l1_LR = np.mean(coef_l1_LR == 0) * 100
        sparsity_l2_LR = np.mean(coef_l2_LR == 0) * 100
        sparsity_en_LR = np.mean(coef_en_LR == 0) * 100

        print(f"C={C:.2f}")
        print(f"{'Sparsity with L1 penalty:':<40} {sparsity_l1_LR:.2f}%")
        print(f"{'Sparsity with Elastic-Net penalty:':<40} {sparsity_en_LR:.2f}%")
        print(f"{'Sparsity with L2 penalty:':<40} {sparsity_l2_LR:.2f}%")
        print(f"{'Score with L1 penalty:':<40} {clf_l1_LR.score(X, y):.2f}")
        print(f"{'Score with Elastic-Net penalty:':<40} {clf_en_LR.score(X, y):.2f}")
        print(f"{'Score with L2 penalty:':<40} {clf_l2_LR.score(X, y):.2f}")

        if i == 0:
            axes_row[0].set_title("L1 penalty")
            axes_row[1].set_title("Elastic-Net\nl1_ratio = %s" % l1_ratio)
            axes_row[2].set_title("L2 penalty")

        for ax, coefs in zip(axes_row, [coef_l1_LR, coef_en_LR, coef_l2_LR]):
            ax.imshow(
                np.abs(coefs.reshape(8, 8)),
                interpolation="nearest",
                cmap="binary",
                vmax=1,
                vmin=0,
            )
            ax.set_xticks(())
            ax.set_yticks(())

        axes_row[0].set_ylabel(f"C = {C}")

    plt.show()


.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 0.413 seconds)


.. _sphx_glr_download_auto_examples_linear_model_plot_logistic_l1_l2_sparsity.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: binder-badge

      .. image:: images/binder_badge_logo.svg
        :target: https://mybinder.org/v2/gh/scikit-learn/scikit-learn/1.8.X?urlpath=lab/tree/notebooks/auto_examples/linear_model/plot_logistic_l1_l2_sparsity.ipynb
        :alt: Launch binder
        :width: 150 px

    .. container:: lite-badge

      .. image:: images/jupyterlite_badge_logo.svg
        :target: ../../lite/lab/index.html?path=auto_examples/linear_model/plot_logistic_l1_l2_sparsity.ipynb
        :alt: Launch JupyterLite
        :width: 150 px

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_logistic_l1_l2_sparsity.ipynb <plot_logistic_l1_l2_sparsity.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_logistic_l1_l2_sparsity.py <plot_logistic_l1_l2_sparsity.py>`

    .. container:: sphx-glr-download sphx-glr-download-zip

      :download:`Download zipped: plot_logistic_l1_l2_sparsity.zip <plot_logistic_l1_l2_sparsity.zip>`


.. include:: plot_logistic_l1_l2_sparsity.recommendations


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_
