

.. _sphx_glr_auto_examples_linear_model:

.. _linear_examples:

Generalized Linear Models
-------------------------

Examples concerning the :mod:`sklearn.linear_model` module.



.. raw:: html

    <div class="sphx-glr-thumbnails">

.. thumbnail-parent-div-open

.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This example compares two different bayesian regressors:">

.. only:: html

  .. image:: /auto_examples/linear_model/images/thumb/sphx_glr_plot_ard_thumb.png
    :alt:

  :doc:`/auto_examples/linear_model/plot_ard`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Comparing Linear Bayesian Regressors</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Computes a Bayesian Ridge Regression of Sinusoids.">

.. only:: html

  .. image:: /auto_examples/linear_model/images/thumb/sphx_glr_plot_bayesian_ridge_curvefit_thumb.png
    :alt:

  :doc:`/auto_examples/linear_model/plot_bayesian_ridge_curvefit`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Curve Fitting with Bayesian Ridge Regression</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This example compares decision boundaries of multinomial and one-vs-rest logistic regression on a 2D dataset with three classes.">

.. only:: html

  .. image:: /auto_examples/linear_model/images/thumb/sphx_glr_plot_logistic_multinomial_thumb.png
    :alt:

  :doc:`/auto_examples/linear_model/plot_logistic_multinomial`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Decision Boundaries of Multinomial and One-vs-Rest Logistic Regression</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Stochastic Gradient Descent is an optimization technique which minimizes a loss function in a stochastic fashion, performing a gradient descent step sample by sample. In particular, it is a very efficient method to fit linear models.">

.. only:: html

  .. image:: /auto_examples/linear_model/images/thumb/sphx_glr_plot_sgd_early_stopping_thumb.png
    :alt:

  :doc:`/auto_examples/linear_model/plot_sgd_early_stopping`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Early stopping of Stochastic Gradient Descent</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="The following example shows how to precompute the gram matrix while using weighted samples with an ElasticNet.">

.. only:: html

  .. image:: /auto_examples/linear_model/images/thumb/sphx_glr_plot_elastic_net_precomputed_gram_matrix_with_weighted_samples_thumb.png
    :alt:

  :doc:`/auto_examples/linear_model/plot_elastic_net_precomputed_gram_matrix_with_weighted_samples`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Fitting an Elastic Net with a precomputed Gram Matrix and Weighted Samples</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Fit Ridge and HuberRegressor on a dataset with outliers.">

.. only:: html

  .. image:: /auto_examples/linear_model/images/thumb/sphx_glr_plot_huber_vs_ridge_thumb.png
    :alt:

  :doc:`/auto_examples/linear_model/plot_huber_vs_ridge`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">HuberRegressor vs Ridge on dataset with strong outliers</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="The multi-task lasso allows to fit multiple regression problems jointly enforcing the selected features to be the same across tasks. This example simulates sequential measurements, each task is a time instant, and the relevant features vary in amplitude over time while being the same. The multi-task lasso imposes that features that are selected at one time point are select for all time point. This makes feature selection by the Lasso more stable.">

.. only:: html

  .. image:: /auto_examples/linear_model/images/thumb/sphx_glr_plot_multi_task_lasso_support_thumb.png
    :alt:

  :doc:`/auto_examples/linear_model/plot_multi_task_lasso_support`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Joint feature selection with multi-task Lasso</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Comparison of the sparsity (percentage of zero coefficients) of solutions when L1, L2 and Elastic-Net penalty are used for different values of C. We can see that large values of C give more freedom to the model.  Conversely, smaller values of C constrain the model more. In the L1 penalty case, this leads to sparser solutions. As expected, the Elastic-Net penalty sparsity is between that of L1 and L2.">

.. only:: html

  .. image:: /auto_examples/linear_model/images/thumb/sphx_glr_plot_logistic_l1_l2_sparsity_thumb.png
    :alt:

  :doc:`/auto_examples/linear_model/plot_logistic_l1_l2_sparsity`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">L1 Penalty and Sparsity in Logistic Regression</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="The present example compares three l1-based regression models on a synthetic signal obtained from sparse and correlated features that are further corrupted with additive Gaussian noise:">

.. only:: html

  .. image:: /auto_examples/linear_model/images/thumb/sphx_glr_plot_lasso_and_elasticnet_thumb.png
    :alt:

  :doc:`/auto_examples/linear_model/plot_lasso_and_elasticnet`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">L1-based models for Sparse Signals</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This example reproduces the example of Fig. 2 of [ZHT2007]_. A LassoLarsIC estimator is fit on a diabetes dataset and the AIC and the BIC criteria are used to select the best model.">

.. only:: html

  .. image:: /auto_examples/linear_model/images/thumb/sphx_glr_plot_lasso_lars_ic_thumb.png
    :alt:

  :doc:`/auto_examples/linear_model/plot_lasso_lars_ic`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Lasso model selection via information criteria</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This example focuses on model selection for Lasso models that are linear models with an L1 penalty for regression problems.">

.. only:: html

  .. image:: /auto_examples/linear_model/images/thumb/sphx_glr_plot_lasso_model_selection_thumb.png
    :alt:

  :doc:`/auto_examples/linear_model/plot_lasso_model_selection`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Lasso model selection: AIC-BIC / cross-validation</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="We show that linear_model.Lasso provides the same results for dense and sparse data and that in the case of sparse data the speed is improved.">

.. only:: html

  .. image:: /auto_examples/linear_model/images/thumb/sphx_glr_plot_lasso_dense_vs_sparse_data_thumb.png
    :alt:

  :doc:`/auto_examples/linear_model/plot_lasso_dense_vs_sparse_data`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Lasso on dense and sparse data</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This example shows how to compute the &quot;paths&quot; of coefficients along the Lasso, Lasso-LARS, and Elastic Net regularization paths. In other words, it shows the relationship between the regularization parameter (alpha) and the coefficients.">

.. only:: html

  .. image:: /auto_examples/linear_model/images/thumb/sphx_glr_plot_lasso_lasso_lars_elasticnet_path_thumb.png
    :alt:

  :doc:`/auto_examples/linear_model/plot_lasso_lasso_lars_elasticnet_path`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Lasso, Lasso-LARS, and Elastic Net paths</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Here we fit a multinomial logistic regression with L1 penalty on a subset of the MNIST digits classification task. We use the SAGA algorithm for this purpose: this a solver that is fast when the number of samples is significantly larger than the number of features and is able to finely optimize non-smooth objective functions which is the case with the l1-penalty. Test accuracy reaches &gt; 0.8, while weight vectors remains sparse and therefore more easily interpretable.">

.. only:: html

  .. image:: /auto_examples/linear_model/images/thumb/sphx_glr_plot_sparse_logistic_regression_mnist_thumb.png
    :alt:

  :doc:`/auto_examples/linear_model/plot_sparse_logistic_regression_mnist`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">MNIST classification using multinomial logistic + L1</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Comparison of multinomial logistic L1 vs one-versus-rest L1 logistic regression to classify documents from the newgroups20 dataset. Multinomial logistic regression yields more accurate results and is faster to train on the larger scale dataset.">

.. only:: html

  .. image:: /auto_examples/linear_model/images/thumb/sphx_glr_plot_sparse_logistic_regression_20newsgroups_thumb.png
    :alt:

  :doc:`/auto_examples/linear_model/plot_sparse_logistic_regression_20newsgroups`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Multiclass sparse logistic regression on 20newgroups</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="In this example, we fit a linear model with positive constraints on the regression coefficients and compare the estimated coefficients to a classic linear regression.">

.. only:: html

  .. image:: /auto_examples/linear_model/images/thumb/sphx_glr_plot_nnls_thumb.png
    :alt:

  :doc:`/auto_examples/linear_model/plot_nnls`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Non-negative least squares</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This example shows how to approximate the solution of sklearn.svm.OneClassSVM in the case of an RBF kernel with sklearn.linear_model.SGDOneClassSVM, a Stochastic Gradient Descent (SGD) version of the One-Class SVM. A kernel approximation is first used in order to apply sklearn.linear_model.SGDOneClassSVM which implements a linear One-Class SVM using SGD.">

.. only:: html

  .. image:: /auto_examples/linear_model/images/thumb/sphx_glr_plot_sgdocsvm_vs_ocsvm_thumb.png
    :alt:

  :doc:`/auto_examples/linear_model/plot_sgdocsvm_vs_ocsvm`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">One-Class SVM versus One-Class SVM using Stochastic Gradient Descent</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="1. Ordinary Least Squares:    We illustrate how to use the ordinary least squares (OLS) model,    LinearRegression, on a single feature of    the diabetes dataset. We train on a subset of the data, evaluate on a    test set, and visualize the predictions.">

.. only:: html

  .. image:: /auto_examples/linear_model/images/thumb/sphx_glr_plot_ols_ridge_thumb.png
    :alt:

  :doc:`/auto_examples/linear_model/plot_ols_ridge`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Ordinary Least Squares and Ridge Regression</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Using orthogonal matching pursuit for recovering a sparse signal from a noisy measurement encoded with a dictionary">

.. only:: html

  .. image:: /auto_examples/linear_model/images/thumb/sphx_glr_plot_omp_thumb.png
    :alt:

  :doc:`/auto_examples/linear_model/plot_omp`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Orthogonal Matching Pursuit</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Shows the effect of collinearity in the coefficients of an estimator.">

.. only:: html

  .. image:: /auto_examples/linear_model/images/thumb/sphx_glr_plot_ridge_path_thumb.png
    :alt:

  :doc:`/auto_examples/linear_model/plot_ridge_path`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Plot Ridge coefficients as a function of the regularization</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Plot decision surface of multi-class SGD on iris dataset. The hyperplanes corresponding to the three one-versus-all (OVA) classifiers are represented by the dashed lines.">

.. only:: html

  .. image:: /auto_examples/linear_model/images/thumb/sphx_glr_plot_sgd_iris_thumb.png
    :alt:

  :doc:`/auto_examples/linear_model/plot_sgd_iris`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Plot multi-class SGD on the iris dataset</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This example illustrates the use of log-linear Poisson regression on the French Motor Third-Party Liability Claims dataset from [1]_ and compares it with a linear model fitted with the usual least squared error and a non-linear GBRT model fitted with the Poisson loss (and a log-link).">

.. only:: html

  .. image:: /auto_examples/linear_model/images/thumb/sphx_glr_plot_poisson_regression_non_normal_loss_thumb.png
    :alt:

  :doc:`/auto_examples/linear_model/plot_poisson_regression_non_normal_loss`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Poisson regression and non-normal loss</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This example demonstrates how to approximate a function with polynomials up to degree degree by using ridge regression. We show two different ways given n_samples of 1d points x_i:">

.. only:: html

  .. image:: /auto_examples/linear_model/images/thumb/sphx_glr_plot_polynomial_interpolation_thumb.png
    :alt:

  :doc:`/auto_examples/linear_model/plot_polynomial_interpolation`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Polynomial and Spline interpolation</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This example illustrates how quantile regression can predict non-trivial conditional quantiles.">

.. only:: html

  .. image:: /auto_examples/linear_model/images/thumb/sphx_glr_plot_quantile_regression_thumb.png
    :alt:

  :doc:`/auto_examples/linear_model/plot_quantile_regression`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Quantile regression</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip=" Train l1-penalized logistic regression models on a binary classification problem derived from the Iris dataset.">

.. only:: html

  .. image:: /auto_examples/linear_model/images/thumb/sphx_glr_plot_logistic_path_thumb.png
    :alt:

  :doc:`/auto_examples/linear_model/plot_logistic_path`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Regularization path of L1- Logistic Regression</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="A model that overfits learns the training data too well, capturing both the underlying patterns and the noise in the data. However, when applied to unseen data, the learned associations may not hold. We normally detect this when we apply our trained predictions to the test data and see the statistical performance drop significantly compared to the training data.">

.. only:: html

  .. image:: /auto_examples/linear_model/images/thumb/sphx_glr_plot_ridge_coeffs_thumb.png
    :alt:

  :doc:`/auto_examples/linear_model/plot_ridge_coeffs`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Ridge coefficients as a function of the L2 Regularization</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Here a sine function is fit with a polynomial of order 3, for values close to zero.">

.. only:: html

  .. image:: /auto_examples/linear_model/images/thumb/sphx_glr_plot_robust_fit_thumb.png
    :alt:

  :doc:`/auto_examples/linear_model/plot_robust_fit`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Robust linear estimator fitting</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="In this example, we see how to robustly fit a linear model to faulty data using the ransac_regression algorithm.">

.. only:: html

  .. image:: /auto_examples/linear_model/images/thumb/sphx_glr_plot_ransac_thumb.png
    :alt:

  :doc:`/auto_examples/linear_model/plot_ransac`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Robust linear model estimation using RANSAC</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Plot the maximum margin separating hyperplane within a two-class separable dataset using a linear Support Vector Machines classifier trained using SGD.">

.. only:: html

  .. image:: /auto_examples/linear_model/images/thumb/sphx_glr_plot_sgd_separating_hyperplane_thumb.png
    :alt:

  :doc:`/auto_examples/linear_model/plot_sgd_separating_hyperplane`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">SGD: Maximum margin separating hyperplane</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Contours of where the penalty is equal to 1 for the three penalties L1, L2 and elastic-net.">

.. only:: html

  .. image:: /auto_examples/linear_model/images/thumb/sphx_glr_plot_sgd_penalties_thumb.png
    :alt:

  :doc:`/auto_examples/linear_model/plot_sgd_penalties`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">SGD: Penalties</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Plot decision function of a weighted dataset, where the size of points is proportional to its weight.">

.. only:: html

  .. image:: /auto_examples/linear_model/images/thumb/sphx_glr_plot_sgd_weighted_samples_thumb.png
    :alt:

  :doc:`/auto_examples/linear_model/plot_sgd_weighted_samples`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">SGD: Weighted samples</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="A plot that compares the various convex loss functions supported by SGDClassifier .">

.. only:: html

  .. image:: /auto_examples/linear_model/images/thumb/sphx_glr_plot_sgd_loss_functions_thumb.png
    :alt:

  :doc:`/auto_examples/linear_model/plot_sgd_loss_functions`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">SGD: convex loss functions</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Computes a Theil-Sen Regression on a synthetic dataset.">

.. only:: html

  .. image:: /auto_examples/linear_model/images/thumb/sphx_glr_plot_theilsen_thumb.png
    :alt:

  :doc:`/auto_examples/linear_model/plot_theilsen`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Theil-Sen Regression</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This example illustrates the use of Poisson, Gamma and Tweedie regression on the French Motor Third-Party Liability Claims dataset, and is inspired by an R tutorial [1]_.">

.. only:: html

  .. image:: /auto_examples/linear_model/images/thumb/sphx_glr_plot_tweedie_regression_insurance_claims_thumb.png
    :alt:

  :doc:`/auto_examples/linear_model/plot_tweedie_regression_insurance_claims`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Tweedie regression on insurance claims</div>
    </div>


.. thumbnail-parent-div-close

.. raw:: html

    </div>


.. toctree::
   :hidden:

   /auto_examples/linear_model/plot_ard
   /auto_examples/linear_model/plot_bayesian_ridge_curvefit
   /auto_examples/linear_model/plot_logistic_multinomial
   /auto_examples/linear_model/plot_sgd_early_stopping
   /auto_examples/linear_model/plot_elastic_net_precomputed_gram_matrix_with_weighted_samples
   /auto_examples/linear_model/plot_huber_vs_ridge
   /auto_examples/linear_model/plot_multi_task_lasso_support
   /auto_examples/linear_model/plot_logistic_l1_l2_sparsity
   /auto_examples/linear_model/plot_lasso_and_elasticnet
   /auto_examples/linear_model/plot_lasso_lars_ic
   /auto_examples/linear_model/plot_lasso_model_selection
   /auto_examples/linear_model/plot_lasso_dense_vs_sparse_data
   /auto_examples/linear_model/plot_lasso_lasso_lars_elasticnet_path
   /auto_examples/linear_model/plot_sparse_logistic_regression_mnist
   /auto_examples/linear_model/plot_sparse_logistic_regression_20newsgroups
   /auto_examples/linear_model/plot_nnls
   /auto_examples/linear_model/plot_sgdocsvm_vs_ocsvm
   /auto_examples/linear_model/plot_ols_ridge
   /auto_examples/linear_model/plot_omp
   /auto_examples/linear_model/plot_ridge_path
   /auto_examples/linear_model/plot_sgd_iris
   /auto_examples/linear_model/plot_poisson_regression_non_normal_loss
   /auto_examples/linear_model/plot_polynomial_interpolation
   /auto_examples/linear_model/plot_quantile_regression
   /auto_examples/linear_model/plot_logistic_path
   /auto_examples/linear_model/plot_ridge_coeffs
   /auto_examples/linear_model/plot_robust_fit
   /auto_examples/linear_model/plot_ransac
   /auto_examples/linear_model/plot_sgd_separating_hyperplane
   /auto_examples/linear_model/plot_sgd_penalties
   /auto_examples/linear_model/plot_sgd_weighted_samples
   /auto_examples/linear_model/plot_sgd_loss_functions
   /auto_examples/linear_model/plot_theilsen
   /auto_examples/linear_model/plot_tweedie_regression_insurance_claims

